Game Theory Meets AI: My Experiment with OpenAI's New o1-Preview Model
As a trained economist, I couldn't resist putting OpenAI's new o1 models to the test with a game theory scenario
As a trained economist, I couldn't resist putting OpenAI's new o1 models to the test with a game theory scenario.
Check out the video below to see the result unfold!
🎭 The Setting: A complex auction between five telecoms companies bidding for use of six mobile bandwidths.
🎮 The Experiment: Understanding if o1 can determine each company's optimal bidding strategies from a game theory perspective, and consequently predict the auction's outcomes.
⚡o1's Performance:
1️⃣ Accurate Reasoning (5/5): Accurately analysed a complex scenario accurately with only a single attempt. Similar attempts with Claude 3.5 Sonnet and GPT-4o resulted in wrong answers.
2️⃣ Depth of Analysis (4/5): Reached the most obvious solution but did not consider other important considerations such as the "winner's curse", where winners tend to overpay due to overestimating the value of the prize.
3️⃣ Time Taken (4/5): Total analysis time took 86 seconds. This may seem long compared to other LLMs. BUT, the outputs were clearly superior, and this whole analysis would have taken me a lot longer to manually figure out. Therefore, no complaints from me!
💡 Potential Business Applications:
The o1 models could potentially be useful in areas such as:
- M&A strategies
- Supply chain optimisation
- Resource scheduling
- Energy market bidding
- Programmatic advertising
🤔 Food for Thought:
While o1's capabilities are impressive, it's important to consider:
- The very "black box" nature of its decision-making process
- The need for human oversight and domain expertise for validation
- Limited weekly usage (30 messages for o1-preview, 50 for o1-mini)
- For most non-math, non-complex logic tasks, models like GPT-4o and Claude 3.5 Sonnet remain more efficient options
This experiment is just the beginning. I'll be exploring o1's capabilities further in science, math, and coding. Stay tuned for more insights!
Have you tried the o1 models? What do you think its impact could be on strategic decision-making? Let's discuss in the comments!
Thank you for reading New World Navigator! I’d love to have you join the community and you can do so by subscribing for free to receive new posts and support this labour of love :)
Justin Tan is passionate about supporting organisations and teams to navigate disruptive change and towards sustainable and robust growth. He founded Evolutio Consulting in 2021 to help senior leaders to upskill and accelerate adoption of AI within their organisation through AI literacy and proficiency training, and also works with his clients to design and build bespoke AI solutions that drive growth and productivity for their businesses. If you're pondering how to harness these technologies in your business, or simply fancy a chat about the latest developments in AI, why not reach out?