Ellen first saw Smallhythe Place when she was riding through from Rye to Tenterden with Henry Irving on her pony and trap and instantly fell in love with the property, so requested to be told if it were ever to become available to buy. Some years later in 1899 a postcard with a Tenterden postmark was sent to Ellen's Chelsea house with the brief message "House for Sale". Later that year she purchased Smallhythe Place and lived there happily for almost 30 years.
For Ellen, Smallhythe Place was a retreat away from her busy acting career in London. She took great joy in escaping to the countryside and tending to her much-loved “daffodilly farm”. It provided Ellen with peace and solace; a complete contrast to city life, and was the place she truly felt at home. After a visit to Smallhythe Place, the critic E.V. Lucas wrote that the house mirrored Ellen’s character: