It comes from Kabbalah. An early source for it is Midrash Hane'elam (a section of the Zohar), which states that "each of the forefathers knew Hashem through his own mirror [i.e., in a way that reflected his own personality]: ... Yitzchak knew Him on the level of gevurah, which is called "the Fear of Yitzchak" (Gen. 31:42), and always feared Him..."
There is also the idea that Eisav inherited Yitzchak's gevurah, but took it to an extreme, becoming not just stern but actually vicious. (By contrast, Yaakov also inherited it, but then went on to blend it with Avraham's chesed ("kindness"), yielding tif'eres ("beauty"), where chesed and gevurah are each utilized in their proper context.)