Josephus agrees that the First Temple was 120 cubits tall on two different occasions. The Second Temple was only half as tall as the First Temple. It stood only 60 cubits or about 10 stories tall. The Mishna states that the Second Temple was later remodeled and 40 cubits were added to it that made it a total of 100 cubits. This corresponds with what King Herod did when he dismantled and re-erected the Second Temple and made it taller.

“Our fathers, indeed, when they were returned from Babylon, built this temple to God Almighty, yet does it want sixty cubits of its largeness in altitude for so much did that first temple which Solomon built exceed this temple nor let any one condemn our fathers for the negligence or want of piety herein, for it was not their fault that the temple was no higher; for they were Cyrus and Darius the son of Hystaspes, who determined the measure for its rebuilding; and it hath been by reason of the subjection of those father of ours to them and to their posterity, and after them to the Macedonians, that they had not the opportunity to follow the original model of the pious edifice, nor could they raise it to its ancient altitude”
Josephus, Antiquities of the Jews, B-XV, C-XI, V.1

“Now the priests and Levites, and the elder part of the families, recollecting with themselves how much greater and more sumptuous the old temple had been, seeing that now made how much inferior it was, on account of their poverty, to that which had been built of old considered with themselves how much their happy state was sunk below what it had been of old, as well as their temple . Hereupon they were disconsolate, and not able to contain their grief, and proceeded so far as to lament and shed tears on those accounts; but the people in general were contented with their present condition; and because they were allowed to build them a temple, they desired no more, and neither regarded nor remembered, nor indeed at all tormented themselves with the comparison of that and the former temple, as if this were below their expectations; but the wailing of the old men and of the priests, on account of the deficiency of the temple. in their opinion, if compared with that which had been demolished, overcome the sounds of the trumpets and the rejoicing of the people.”
Josephus, Antiquities of the Jews, B-XI, C-IV, V-2.

“Who is left among you that saw this house in its first glory? and how Jewish Mendo you see it now? is it not in your eyes as nothing? Yet now be strong, O Zerrubbavel, says the LORD; and be strong, O Yehoshua, son of Yehozadaq, the high priest; and be strong, all you people of the land, says the LORD of hosts”. HAGGAY 2:3-4 JB

“The Sanctuary was 100 Cubits Square and a hundred cubits in height..”
Mishna Middoth, 4.6

“Who is left among you that saw this house in her first glory? And how do ye see it now? Is it not in your eyes in comparison of it as nothing.”
Haggai 2:3-4 KJV

Most of the inside measurements of the First Temple are found in 1 Kings & 2 Chronicles. Although the First Temple had been destroyed Ezekiel was brought back to Jerusalem and was given the outside measurements of that destroyed Temple and courtyard, which Solomon had built. He also confirms many of the earlier inside measurements, which were previously written down in I Kings and II Chronicles.

The First Temple was erected symmetrically within the 500-cubit courtyard. It was not located off centered to the outer courtyard as the Second Temple was.

The 20 story tall Temple of Solomon stood majestically in the center of the courtyard. There were three tall gates, which entered into the outer courtyard. Each of the three gates had seven steps. And there were also three identical gates which entered into the inner courtyard. Each of these gates had eight steps which meant the inner courtyard was on a raised platform. The outer gates were mirror images of the inner gates. Each gate stood 10-stories tall.

An often over looked fact when the Second Temple was built it was not erected on the original foundation stones of the First Temple. A new set of foundation stones was laid for the Second Temple which were to the North West of the original Temple. Unlike the First Temple the Mishna confirms that the Second Temple was built off-centered to the square by at least 35-50 cubits to the NorthWest. Which means the old foundation stones might have remained in order to someday rebuild the Temple which Ezekiel promised that they would. Both Temples were located upon the raised platform of the inner courtyard , which was centered within the original 500-cubit courtyard. But approximately 35-50 cubits apart.
Because the Second Temple was not built on the original foundation this created uneven spaces in the outer courtyard. Historians have noted this fact.

“The Temple Mount measured five hundred cubits by five hundred cubits. Its largest (open) space was to the south, the next largest to the east, the third largest to the north, and its smallest (open space) was to the west. …… The free space to the south was 265 by 500 cubits, to the east of the inner rectangle was a space of 115 cubits; to the north was a space of 100 cubits; while west of the Temple structure there was only 63 cubits.”
Mishna Middoth, 2.1, & footnote 9.

“In the first year of Koresh (CYRUS) the king the same Koresh the king made a decree, Concerning the house of God at Yerushalayim; and let the house be built, the place where they offer sacrifices, and let the foundations thereof be strongly laid; its height 60 cubits, and its breadth 60 cubits.”
‘EZRA 6:3 JB

“From the first day of the seventh month they began to offer burnt offerings unto the LORD. But the foundation of the temple of the LORD was not yet laid. They gave money also to the masons, and to the carpenters and food, and drink, and oil …” ‘EZRA 3:6 JB