173: Things to test when editors use AI

In my experience, creating summaries of articles, and creating images to accompany articles are the best use of AI right now

The Daily Maverick learned some interesting things while experimenting with ChatGPT version 3. I’ll provide the link below.

I don’t believe their specific results are relevant to ChatGPT 4, which has fixed several of the problems with version 3, but they do identify a series of things that your should test if you want to try AI yourself. In the list below, I’ll include the things they tried, plus my own recommendations.

For article summaries …

* Do you want the summary to include every point in the article, or do you want it to entice the reader to read the full article?
* Is the summary accurate? Does it capture nuance and context?
* Do you want the summary to stay within a certain character, word, or sentence length?
* Does the quality of the summary vary based on the structure of the article – e.g., a pyramid style article might get one kind of summary, while a story with a flowery intro might get another sort altogether.
* Should summaries vary in tone or style for different sorts of articles, e.g., news, opinion, or feature stories.
* Can the summary duplicate the tone of your publication? And does that matter?

Then, of course, are the well-known problems of hallucinations and the simple question whether the summaries are any good.

Remember that you may be able to correct problems you identify by adjusting the prompts you use.

If you’re going to experiment with something like this, remember Gall’s law – that is, start with something simple that works! And make sure you take the writer’s or editor’s workflow into consideration. It’s no good to build something they won’t use.

If you find a way to create good summaries using AI, the next question is how to use them.

On my blog I put the summary right up front. Some websites have found that increases engagement with the article, but my reason is simpler. I’m not writing a mystery novel, and if the summary is good enough, so be it.

Here are some other ideas.

Use the summaries in the mobile app, as a popup, or sidebar. You could even create a completely new interface that just has the summaries. Sometimes that’s all people want.

You can try text summaries or summaries expressed in bullet points.

In addition to summaries, AI can be used to create translations of your articles.

I believe you should label things that are generated by AI, unless it’s fairly obvious. For example, I use AI-generated images on my site, but I think it’s obvious they’re from AI, so I don’t label them as such – although I do have a note on the page to explain how I use AI on the site.

Links

Daily Maverick experiments with in-house AI solutions while learning from its editors
https://www.inma.org/blogs/ideas/post.cfm/daily-maverick-experiments-with-in-house-ai-solutions-while-learning-from-its-editors