A campaign group says the they're not just missing out on books, but the building also houses the local archives