Due to advances in natural language understanding, chatbots have become popular for assisting users in various tasks, for example, searching. Chatbots allow natural-language queries, which can be useful in case of complex information needs, and they provide a higher level of interactivity by displaying information in a dialog-like format. However, chatbots are often only used as auxiliaries for a graphical search user interface. Thus, they must be engaging and usable so that users both want to and able to use them. In this study, a chatbot-based and a website-based search interface were compared in terms of engagement and usability. Engagement was measured using the User Engagement Scale; think-aloud protocol and a questionnaire were used to assess usability. Behavioral measures were used to triangulate data. Findings indicate that the usage of the chatbot did not lead to a higher level of engagement, moreover, its usability was lower compared to the website-based search interface.