Mantua was once surrounded by four lakes formed by the river Mincio. Not far from the island on which the city was built was another island, in Medieval times called Teieto (then abbreviated to Te).
This was connected to the south walls of the city by a bridge. There are two likely theories regarding the meaning of the word Teieto: it may derive from tiglieto, linden tree wood, or tegia, from the Latin attegia, meaning hut.

The island, which would have been verdant and quiet, soon became a place of leisure for the Gonzaga family. Numerous documents inform us that it was already in use by the mid fifteenth century.

At the beginning of 1500 Francesco II Gonzaga, husband of Isabella d’Este, had a house and stables for his thoroughbred horses built here. Traces remain of a prestigious building with painted walls in the area below the roof of the present Palazzo. One of the frescoes is dated 1502.