Assuming the question is "Cuándo vamos a la biblioteca", this is asking "When are we going to the the library?" So, the response would likely be something like one of these: Vamos a biblioteca a las cuatro. Vamos a biblioteca el miercoles.
Or, more simply, something like these: Vamos en dos horas. Vamos mañana. Vamos el viernes.
