The Gray Quarter problems are not so simply reduced to 'Ulfric is a racist bastard', Ulfric is preoccupied with a war and doesn't want to expend what little resources he has on a constituency offering nothing in return, beyond that he hasn't disadvantaged them especially. Rolff Stone-fist and Agrenor Once-honoured, a drunk and a hobo respectively, are hardly representative of Windhelm as a whole. Even Brunwulf says most nords are welcoming. Galmar explains the 'with us or against us' dialogue and you can encounter random Dumner farmers who support Ulfric.
The most vocal complainant is Ambarys Rendarr and even other dunmer comment that he's too bitter and goes overboard.
The book Dunmer of Skyrim is a pretty insightful glimpse into how the Snow Quarter transformed into the Gray Quarter. Several dunmer residents place at least some responsibility for their situation on themselves. Suvaris admits the Dunmer in Windhelm don't support Ulfric's cause and wish to remain neutral in the war.
Despite that, Windhelm has an elven bourgeousie not only comparable with other big cities but perhaps even bigger, and many dunmer who are not complaining so bitterly.
Niranye: Altmer, businessowner, living outside the Gray Quarter.
Arivanya: Altmer, co-owner of Windhelm stable, lives outside Gray Quarter (no anti-Ulfric dialogue)
Ulundil: Bosmer, co-owner of Windhelm stables (cheerful dialogue, no mention of racism)
Belyn Hlaalu: Dunmer, landowner, employs nords (derides Windhelm's dunmer for their attidude)
Aval Atheron: Dunmer, business owner, lives in Gray Quarter and is somewhat btiter but not overly so.
Faryl Atheron: Dunmer, farm-worker, lives in Gray Quarter and is not bitter, blames dunmer for situation and says Dunmer are ashamed of working for nords.
Suvaris Atheron: Dunmer, manager of Shatter-Shield office, lives in Gray Quarter, angry, admits Dunmer don't support the civil war, want to remain neutral.
Revyn Sadri: Dunmer, business owner, Gray Quarter, blames dunmer for situation.
Idesa Sadri: Dunmer, nanny to Grimwar Cruel-Sea, lives with Cruel-Seas. No bitter dialogue.
Ambarys Rendar: Dunmer, business owner, Gray Quarter, vocal protestor
Malthyr Elenil: Dunmer, shop assistant, Gray Quarter, no dialogue about racism
Luaffyn: Bard, Candlehearth Hall, no dialogue about racism
There's Voldsea Giryon but assuming she's not a permanent resident of Windhelm I'm not including her, but note she isn't bitter, and is first mate to a Nord sea-captain.
8 out of 12 elves are self-employed, of which 9 are dunmer. Of the rest, only 3 can be argued are actually 'working class' - Faryl, Idesa and Malthyr - none of which complain about nords. Suvaris is a business manager for the Shatter-Shields, therefore not 'working class' though she's the next loudest protestor after Rendarr.
Now, compare with Riften with the next largest dunmer population and no complaints of racism, where dunmer seem to have integrated satisfactorily.
Indaryn: Dunmer, Manager of Black Briar Meadery
Niluva Hlaalu: Dunmer, janitor of Black Briar Meadery, working class.
Tythis Ulen: Dunmer, janitor at Bolli's fishery, working class.
Marise Aravel: Dunmer, business owner, lives in the slums
Brand-Shei: Dunmer, business owner
Synda Llanith: Dunmer, wife of Dravin.
Dravin Llanith: Dunmer, co-landowner.
Romlyn Dreth: Dunmer, busboy at Black Briar meadery, working class.
Dinya Balu: Dunmer, priestess of Mara
(There's also the two skooma dealers who are dunmer and one odd thief in the Thieves Guild that I'm not including)
3 out of 9 are working class, 2 live in the slums. 1 landowner (+wife), 1 business manager, 2 business owners, 1 priestess.
So not that different from Windhelm.
As a whole Windhelm is poor and miserable and I'm not denying the racial tension but it's not solely Ulfric's fault.
On the argonians, I would make an argument but Brunwulf, the great civil rights champion who curiously (perhaps conveniently) doesn't espouse the persecution of nords at the hands of Imperials and Thalmor, doesn't uplift them either, apparently 'for their safety' but presumably because they're thieving skooma addicts. It's the Shatter-Shields, Stormcloak supporters who give them employment (and Torbjon is even easily persuaded to raise their wages despite the fact that they admit stealing brazenly from him).
Khajiit, no one allows Khajiit caravans inside the cities. No reason is given but I suspect it's because of Anequina and Pelletine's alliance with the Dominion.
This doesn't mean that Ulfric is perfect and rainbows shoot out of his ar*e, perhaps he should make a bigger effort reaching out to them despite their reticence and refusal to integrate but Brunwulf's opinion of him is unfounded and not supported by facts. Brunwulf strikes me as a broken man whose lost his appetitde for war and wants to redeem himself. Survivor's guilt methinks.