The castle is named after St. Hilarion, a hermit monk who fled from persecution in the Holy Land and lived and died in a cave on the mountain. Later in the 10th century the Byzantines built a church and monastery here. The castle was originally built in the 7th century as a watch tower to give warning of approaching Arab pirates.