W3C took care of that in 1997 explaining how frames should be implemented in "Implementing HTML Frames":
Any frame that attempts to assign as its SRC a URL used by any of its ancestors is treated as if it has no SRC URL at all (basically a blank frame).
Iframe recursion bug/attack history
As kingdago found out and mentioned in the comment above, one browser that missed to implement a safeguard for this was Mozilla in 1999. Quote from one of the developers:
This is a parity bug (and a source of possible embarrasment) since MSIE5 doesn't have a problem with these kinds of pages.
I decided to dig some more into this and it turns out that in 2004 this happened again. However, this time JavaScript was involved:
This is the code, what causes it: <iframe name="productcatalog" id="productcatalog" src="page2.htm"></iframe> directly followed by a script with this in it: frames.productcatalog.location.replace(frames.productcatalog.location + location.hash);
...
Actual Results: The parent window gets recursively loaded into the iframe, resulting sometimes in a crash.
Expected Results: Just show it like in Internet Explorer.
Then again in 2008 with Firefox 2 (this also involved JavaScript).
And again in 2009. The interesting part here is that this bug is still open and this attachment: https://bugzilla.mozilla.org/attachment.cgi?id=414035 (will you restrain your curiosity?) will still crash/freeze your Firefox (I just tested it and I almost crashed the whole Ubuntu). In Chrome it just loads indefinitely (probably because each tab lives in a separate process).
As for the other browsers:
- In 2005 Konqueror had a bug in it's safeguard that allowed to render iframes one inside another (but it seems that somehow it wasn't freezing/crashing the whole app).
- IE6, Opera 7.54 and Firefox 0.9.3 are also reported to be susceptible to attacks basing on iframe recursion.
You are an amazing person @Konrad
Well thank you @kingdango. This is a great question, I ended up doing a research on "iframe recursion bug/attack" history, check out my updated answer.
@Konrad as I illustrated in my answer below an iframe recursion attack should still be possible today with all versions of IE - that is if one can exploit the generic crash I found.
You can create this behavior in modern browsers, too -- it just requires JavaScript: e.g.
frame.src = 'http://0.0.0.0:9922/index.html?' + new Date().getTime()