The reality of this trope is far more complicated than the trope itself.
To start with, Nazism was, from beginning to end, a populist, grassroots movement, and German aristocrats were opposed to the involvement of the common people in politics as a matter of principle. The Nazis held mass rallies and recruited ordinary Germans into paramilitary organizations like the SA and the SS; noblemen loathed the idea of mass rallies (or mass-anything, really), and were genuinely worried about the fact that the Nazi paramilitary organizations did not recruit their officers from the ranks of the aristocracy, as the German Army had traditionally done. And, of course, the more political power Hitler concentrated in his own hands, the less power was left to go around for everyone else.The Nazis also sought to not merely compete for political power, but to replace traditional focuses of loyalty; in effect, Nazis and the Aristocracy were competing for the same space and in that sense were more natural adversaries than natural allies. Indeed, one of the Nazi movement's early targets were the aristocratic upper-class Prussian generals who had led Germany to defeat in World War I.
Hitler capitalized on popular disgust with the aristocratic leadership of Wilhem's Imperial Germany, which they scapegoated as incompetent generals who had signed the Treaty of Versailles even though the loyal German army was "winning". This stems from the technicality that the aristocratic generals surrendered before the front lines had actually been pushed back into Germany itself - when they'd exhausted their last reserves of troops in the last desperate offensive of 1918, and with more American reinforcements coming in every day, they realized that defeat was inevitable and it made no sense to keep fighting. Many Germans were so demoralized by the defeat that they defiantly refused to acknowledge this.
So, even though many among the German and Austrian nobility partially or wholly agreed with Nazism's assessment of "lesser races" and were eager to re-assert German military strength in Europe, they generally resented Hitler's appeal to the masses and lack of regard for tradition. Some of them also felt that Hitler was incompetent, and that his war strategy was disastrous, and although some aristocrats had low opinions of non-Germans, most were thoroughly disgusted by Nazi racial extermination policies. As a result, aristocrats were disproportionately involved in military plots against Hitler.
At the same time, Nazis did woo the nobility - with considerable success. By 1938, a fifth of all SS officers were noblemen. High-born noble families were particularly prone to Nazism: between one third and half of all eligible people with princely titles joined the Nazi party. And, of course, the Nazi Wehrmacht was built out of the earlier German armies, so they inherited all of the officers wholesale, including a bunch of aristocratic officers, natch.
There are many reasons that this trope keeps recurring, but some of the main factors include confusing Imperial Germany with Nazi Germany, simple prejudice against aristocracy, a desire to include cool titles and ancestral castles, and assuming that everyone who opposes Democracy does so for exactly the same evil reasons. Another problem is that superficially, the Nazi ideology of "Racial Superiority" sounds like something that ought to dovetail neatly with aristocratic superiority. Unfortunately, writers who focus too much on "official" Nazi ideology tend to miss the fact that the popularity of the Nazi party was also driven by envy, resentment, and fears of inferiority. At the same time, Himmler did see the endogamous practices of the nobility as a model of eugenics.
The reaction to a trope can become a trope itself. Of course, Imperial Germany lead to Nazi Germany, and the German Army fought a genocidal war of extermination for the Third Reich. Both of this rather obvious points have been de-emphasized in - not merely German - public discourse, thanks in part to the early entrance of post-war West Germany into the Western bloc.
To some extent, this trope also applies to the British nobility. A small number of the British aristocracy were sympathetic to the Nazis,. Unfortunately, the handful that were sympathetic were also the ones everyone remembers, often because they were very good at marketing themselves. This means that there are a lot of Britons out there who are wrongly convinced, and often convinced beyond dissuasion, that the aristocrats were all Nazis. "Utter hogwash" doesn't even come close to describing how insanely ludicrous this astounding theory is.