The site has been empty for the past 20 years with many saying it's a wasted opportunity for the town.