Actually, Adam Smith wrote about this in The Wealth of Nations back in 1776.
Most of the wealth that Spain and Portugal extracted from Latin America was in the form of gold and silver, and as Smith himself proved, actual national wealth is not measured in how much precious metal a country has stockpiled. The problem was, in those days everyone thought it was. So they kept hoarding gold and silver, all the while continuing to get poorer and poorer and not understanding the economic mechanisms involved.
Spain and Portugal simply never made the jump to true industrial countries, at least, not until the 20th century. They made very little that anyone wanted to buy, and so they could do nothing with their precious metals except use them up to buy goods from other countries. This did not grow their economies, so they stagnated.