Professor tested AI tools for summarization and research

Indicator: “Hilke Schellmann, an assistant professor of journalism at New York University, tested AI tools for summarization and research. Four LLMs were tasked with summarizing local government meetings. They performed well when asked to generate short summaries but were “surprisingly poor” at longer overviews. Schellmann also tested four summarization tools by asking them to generate a literature review for a scientific topic. The results were “underwhelming and in some cases alarming.”