This semester we’ve introduced a chatbot on two UOC courses. It’s a pilot project in response to an ambitious long-term goal: to provide students with assistance 24/7, all year round.
There’s a long way to go and this year we’ve only taken the first step. But we’ve already learnt a few things.
One: you need a team to love the chatbot
A chatbot needs constant training, so it can give increasingly better answers and adapt to new questions that come up. This requires a team of at least two people with different roles, working on it constantly: an expert in the subject –law, economics, communication, etc.– and an expert in chatbots.
The expert in the subject should create the knowledge base on which the chatbot works and keep it updated as necessary. Obviously, the expert cannot anticipate all the questions students will ask, so they will have to add questions and answers periodically.
The chatbot expert must be able to analyse and interpret how the chatbot is working. They must know which questions are answered satisfactorily, which aren’t, and which go unanswered. They must also be able to modify its settings for the best possible performance.
Two: there can be no “go to the next window”
Online university students know nothing about windows. If there is a chatbot in the classroom, they’ll ask it questions about course content, yes, but they’ll also have questions about the classroom mechanics, such as what format their assignments should be in, how they will be assessed and what will happen if they fail to take part in a mandatory debate.
When we develop the chatbot, we need to take all this into account. So, even if the information comes from very diverse sources, we can’t say to the student: “go to the next window”.
The solution is for the chatbot to feed off different knowledge bases. That implies two things: the chatbot will be more complex and several experts will be required to maintain the various knowledge bases.
Three: the more, the better
To train a chatbot we need data. The more, the better. If a chatbot receives a few dozen queries in the course of a semester, it’ll be tougher to train than if it gets hundreds.
The solution might be to have the chatbot as a front end in the classroom, and have the teacher intervene only when the chatbot is unable to answer a student’s question.
If we do that, we run the risk of “hiding” the teacher and dehumanizing the classroom in the student’s eyes. That decision should be taken very carefully, offsetting the disadvantages –dehumanization of the classroom, two levels of answer–, and leverage the advantages –anonymity of questions, immediacy of answers.
Another solution could be to break down the barrier of classrooms and courses, and develop chatbots for broader areas of knowledge, running the risk of losing accuracy in the answers or of needing to ask the student follow-up questions in order to refine the answer.