The graduate chemist behind me says that there's no obvious connection between azo dyes and salicylates, but that cross-tolerance and allergic reactions don't necessarily have much to do with structural similarity. (Sometimes a person will react to a functional group that's quite a small part of the overall chemistry.)

She then pulled out her copy of the Merck Index, which produced azomycin and azosulfamide (antibiotics), and azosemide (diuretic), all to emphasize that intensely colored molecules are not always just dyes. Though food coloring is intended to be just dye, and not do anything else.

[Said graduate chemist is my beloved adrian_turtle. She wrote most of this comment.]