At Convergence Strategy Group, we are continually refreshing our skills, learning new things, questioning, and growing. Whether through mandated continuing education to maintain my AICP credentials, industry conferences, podcasts, books, or news articles, we are constantly expanding our knowledge base. That brings me to AI, the most hyped-up tool I can recall in some time. With a bit of exposure to AI over the years, I recently thought it time to dig in and learn more about AI in its current form. My general conclusion – while AI seems to have a great deal of potential, it is nowhere near capable of replacing educated, informed, and highly capable humans.
I recently ran a series of prompts through a few different AI platforms: Microsoft’s Bing (coupled with Open AI’s GPT-4), Google’s Bard, and Grammarly. I asked each of these platforms to write a blog post for me (it would be great if AI could take some of my workload and free up my time for other tasks!), and what I received was certainly not worthy of being posted to the Convergence Strategy Group website. The AI platforms produced blog posts that read like they were written by a kid making use of Google’s search engine, but with no understanding of the real issues and most importantly, no ability to separate truth and facts from falsehoods, misinterpretations, and simply bad information. For instance, I gave the platforms the following prompt:
write me a brief blog post for the Convergence Strategy Group website, convergencestrat.com, which discusses the prevalence of gray market gaming machines in the U.S., and recent legislation to shut down this market through legalization of VLT machines.
What I received was, upon first glance, a passable if not deeply researched post on the topic. All three AI produced eerily similar 4th grade level reports on gray market machines, briefly referencing the American Gaming Association’s 2022 report on the topic, and then giving examples of states that have attempted some sort of legislation to shut down the market. Had I not known anything about gray market machines, or VLTs, or the current status of bills before various legislatures, I might be fooled into believing what the AI wrote. However, this is our wheelhouse at Convergence Strategy Group and I know better. The AI missed the most recent, verifiable, and reliable research and reports on gray machine markets. For the states which have recently acted or are currently considering related legislation, AI got the details wrong. It also included states that, to our knowledge (and backed up by actually tracking the bills!), haven’t introduced any such legislation. The 4th grader turning in a report like this would likely receive at least a passing grade for correct grammar usage and attempts at research, but anyone with more than a 4th grade education would receive an “F.”
Now, I never really expected to get AI to do my writing for me, but this was pretty disappointing. I couldn’t even use these essays as a starting point for writing something more in-depth, as every single point had to be checked and verified or debunked, taking me more time than actually writing a piece myself. That said, I remain hopeful for the possibilities of AI. Perhaps one day we’ll be able to type in the most complicated and multi-faceted questions and receive reliable answers from our computers, but until then we’ll just keep on learning and answering those questions ourselves.
Suzanne P. Leckert, AICP