ChatGPT observations

What is it useful for?

Synthesizing useful responses to questions by cross-referencing multiple sources. This saves a lot of time when researching unfamiliar topics and can get you up to speed quickly. It can help you discover what questions to ask and can give useful responses to 'dumb' questions. A search engine cannot do this as it can only return links to relevant sources. The job of finding the relevant parts of the returned documents, cross-referencing and combining them is normally a time-consuming, manual task.

Sometimes it gives responses that sound very naive. This is to be expected. It's like somebody who has read lots of books who can regurgitate knowledge, but has not real-world experience. Reliability of results is still suspect. Reading the sources is still required to sanity-check responses.

Generated code is sometimes subtly incorrect. With some iteration, it can often fix these problems itself. You still need a competent programmer to judge the code and know what to prompt in order to iterate on the code to improve it. This could be improved if the AI could run code through a compiler, read the error messages and fix the code itself before returning it. This is how human programmers work anyway. We don't always write correct code on the first attempt.

It's an excellent tool for discovering APIs and learning how to use them. This can save a lot of time. Ask what APIs or libraries are available on a given platform for the task you are trying to achieve and it will give good answers. Ask for example code to do certain things and it will generate it. It can generate good boilerplate code for any task you can think of.