The Loebner prize is an event held yearly since 1990 to determine if computers have become smart enough to fool humans. It is a watered down form of the Turing test, the AI has to fool only 30% of 12 human judges to win. This year the chatbot called Elbot from Artificial Solutions fooled three judges, only 25%. You can play with it yourself here. An overview article has a transcript of the interview. The experts appear to be technologists rather than ordinary people.
I have worked with applications of chatbots for marketing and information style interactions. People have become very used to search. So it is not clear you want to use natural language queries, with all of their complexity, as an interaction method with a system. Still you may want to 'fool' someone, produce the same intelligence, cadence, humor, flexibility etc. of a human conversation so you could not tell it was a non-human. That might make you feel good thinking that the company you were talking to was using expensive people ... until most companies did. Then you might feel ... fooled. More commentary.