Penshurst began as a 14th century manor house of Sir John de Pulteney but is probably better known for its Tudor and Elizabethan associations with the Sidneys, who were given the property by Edward VI in 1552. Their descendants still occupy the house and estate to this day.

During Tudor times the house was visited by Henry VIII in 1519, then held by Edward Stafford, 3rd Duke of Buckingham. The duke was executed two years later for treason and the house came into the possesion of the crown. In 1552 Edward VI gave the property to Sir William Sidney, which then passed to his son Sir Henry Sidney in 1554. Sir Henry was married to Mary Dudley, sister of Robert Dudley, favorite of Elizabeth I, who visited the house during her reign. Sir Henry's son, Sir Philip Sidney, the famed Elizabethan poet and courtier, was born at Penshurst in 1554.