Tamas Kokeny: Designing for Token Economics: Practical AI Cost Strategy and Experimentation - Productized 2025 TalkInTen

Product Agility Podcast — Live from Productized, Lisbon

We're delighted to be back at the outstanding Productized conference in Lisbon — a world-class event that brings together product leaders, designers and builders to share the latest thinking in product strategy and org design. We're honoured to partner with Productized and grateful to Bobcats Coding for making this Lisbon mini-series possible.

This short episode features a rapid-fire conversation about token economics, AI billing and the product design choices that change unit economics and route-to-market decisions.

Key topics discussed

• What token economics means for AI products and pricing.
• Agentic AI vs chat interfaces: complexity, cost and design trade-offs.
• When to optimise for cost vs when to prioritise learning and speed-to-market.
• Splitting problems into deterministic subproblems to reduce AI calls and cost.
• Essential skills: strategy, experimentation and logical problem decomposition.

[00:00:00] - Opening from Lisbon: Quick intro and why Productized is a must-attend conference for product leaders.
[00:01:30] - Why token economics matters: What founders and PMs should expect on their AI bill and its impact on pricing strategies.
[00:04:20] - Agentic workflows vs simple chat: How multi-agent or generative workflows multiply requests and costs.
[00:09:10] - Practical optimisation: Split problems, remove deterministic work, and avoid premature over-optimisation.
[00:13:55] - Skills to invest in: Strategy, experimentation, logical decomposition and cheap experiments.

Guest bio:

Tamas Kokeny — Co-founder, Bobcats Coding. Tamas leads Bobcats, a Budapest and Lisbon-based digital product studio specialising in AI engineering and end-to-end product development. He runs workshops on token economics and builds practical guidance to help teams design cost-effective AI products.

Thanks to Bobcats Coding for sponsoring our Lisbon sessions — check their AI Economics guidebook and resources at bobcatscoding.com.

Host Bio

Ben is a seasoned expert in product agility coaching, unleashing the potential of people and products. With over a decade of experience, his focus now is product-led growth & agility in organisations of all sizes.

Stay up-to-date with us on our social media📱!

Ben Maynard

🔗 https://www.linkedin.com/in/benmaynard-sheev/

🐦 https://x.com/BenWMaynard

💻 https://sheev.co.uk/

Product Agility Podcast

🔗 https://www.linkedin.com/company/productagilitypod/

💻 https://productagilitypod.co.uk/

🖇️ https://linktr.ee/productagility

Listen & Share On Spotify & iTunes

• Spotify - https://open.spotify.com/show/0lkwAYJzVSuk5zfJ1vIDZq?si=4c691fb12f124a56
• iTunes - https://apple.co/3YvTX8p

Want to come on the podcast?

Want to be a guest or have a guest request? Let us know here https://bit.ly/49osN80

Transcript

Welcome to the Product Agility Podcast where we explore the ever changing world of product leadership and org design, helping you navigate complexity and build better outcomes for your people and your customers. This week we're coming to you live from Lisbon for the third year in a row at the Productize conference where I'm grabbing 10 minute conversations with product thinkers, leaders and innovators from around the world. These quick fire chats are all about what's shaping our industry right now, from AI and product strategy to the human side of building great products. Now a huge thank you goes out to Bobcatz Coding for making this Lisbon series possible. Bobcats is a Budapest and Lisbon based digital product studio specializing in AI engineering and end to end digital product development. They're also on a mission to educate the market, exploring a new topic every six months and this fall is no exception. Their latest AI Economics guidebook is out now and you can download it for free@bobcatscoding.com now here's your talk in 10. I'm here with Tamash Tamash, who is the co founder of Bobcat's Coding, the wonderful sponsors for our time here in Lisbon at Productised. We've taken this one outside because why not? We can do. The conference centre is buzzing. I think everyone is probably in their workshops now, so we're taking this opportunity to talk about a topic which until recently was a new one on me and it's something which you are here to talk about to the wonderful people at Productised. I won't take your thunder, Tamash. What is the topic you're talking about? So thank you for having me. We're going to hold a workshop about token economics. Token economics? I think it's a really broad topic. In this event we're going to focus on basically what can you expect about your AI bills on the end and how can you optimize it. But I think it's also a broad topic from the point of view of business models and so on. So it's a. Yeah, it's a hard topic and it's also a new topic. It's not easy. No, I mean it's a funny one because you simplified it so well then by saying you know, it's about what you'll build. But you also alluded to the fact that it's a much bigger topic because you know what you're being billed and how much things are costing is a direct correlation on how you devise your pricing strategy and your route to market and who you're going to target. So like, because of that, then how are you seeing that the token economics are changing the way that people are designing their systems, whether AI native or not? Yeah. So I think it's tricky because there is the easy part that there are certain type of models, certain type of hosting, and after a certain scale it makes sense to change to a certain type of hosting. For example, it's easier in the beginning to pay for OpenAI and then figure out how you're going to self host something or something like that. And also it's also an easy question that first you experiment with an expensive model that does everything and then you figure out how you can use cheaper and cheaper models for the same thing. But I think what's the hard part? That it varies a lot based on the feature itself. So if you have a simple chat interface, then yeah, you're going to have a cost based on a chat message and based on the length of the conversations. But if it's a more complicated agentic workflow and multiple agents are talking to each other, or you have something, a generational workflow where you have a good amount of sub processes that would do something. And also usually what's the hard part, if you use some kind of feedback loop, so something generates something and also another flow is validating it, then it's not just okay, this many input tokens, that many output tokens, and that's it. But it varies a lot based on the feature. Yeah, and I suppose that's the one thing which maybe helps people understand in some ways the impact of agentic AI versus just like what lots of people assume is AI, which is the user to computer interface of having a chat window. We're talking about magentic AI. You're letting them free to have the conversations effectively to get the job done. Exactly. And then with that comes, unless you're capping it in some way that could increase exponentially, especially if you've not designed it in such a way that actually knows when to stop or it can't get an answer. A good example for that is Khan Academy. Yeah, they even, I think more than two years ago they released a chatbot that's a tutor that helps with math problems and they were even using GPT 3.5. So it's a really old model. But even with 3.5 they managed to make the math correct because they had an internal loop that always re evaluated the math. So they made sure that there is no hallucination. So from the point of view of the user, it was a normal chat interface. But in the Background, a shitload of things happened. It's not just a single fire and run and you get the output. And it's a good example for that that maybe 10 or 20 or 20 something requests went to OpenAI for a single message. Yeah, but they were low cost. Yeah, but it was the cheapest model so it still made sense. But yeah, you really need to figure out the exact use case. Well, it's a big design consideration, isn't it? Because maybe the effort, the investment that you put into building that kind of the self checking kind of flow there for the Gentic AI maybe actually would be cheaper just to go to 5 or 5 mini. Yeah, exactly. And you just wait two months and the whole infrastructure that you have built and it can be way more expensive than the AI bill if you pay five developers to create a really complicated agenting flow and a new reasoning model comes up that does the whole thing in a fire and evaluate loop. Yeah, it's tricky, but it really reminds me of when we're talking about AI generally speaking and people are fearful of what it can do right now, maybe where it's going to go. But if we're honest about it in this example, just because you say you would like to use AI and use some kind of token based model as your pricing strategy, as your commercial model, isn't then instantly going to turn the people within the organization into experts in how to design all of this. So there is a lot of learning that needs to happen, I guess. So you can invest all the money to build this lovely flow to validate it and within two months you realize it would have been cheaper just to go on to GPT5 mini and use that. So do you think that the token economy and the way that it's evolving is helping or hindering innovation in people's ability just to kind of just to try stuff? Yeah, exactly. I think in many of the cases when a new client comes to us, the first question is financial feasibility and create a financial feasibility experiment to figure out. And my first answer is always we should not do that and we should wait at least half a year or something like that, figure out what's the established workflow that really works and what's the feature that really something that creates value for the end users. And then because in half a year, that's a decade in AI time, wait for that, then it's totally all right to use the most expensive model and so on and then figure out okay, what's going to be the economy of scale, what's going to be the unit economy of the whole thing and so on, rather than right away start early optimization because yeah, that's going to hinder innovation. Yeah, okay. I'm just going to keep an eye on the time. We've got a few minutes left and I almost want to loop back a little bit on what we're talking about before and particularly that maths example where they were optimizing for output quality while still, but also kind of duly optimizing then for minimal or reduce token usage. How to give some advice to an organization who's trying to figure out how to kind of strike that balance and deal with those difficult design considerations. Given that maybe they haven't gotten skills internally, what advice would you provide? I think the usual thing that I've seen is the best is to figure out sub problems because usually the naive solution is to pay for the biggest model. Put every context that you can find there and then on the end pray that the output is going to be good enough. And usually there is a way cheaper and also way more accurate solution when you're able to split up your problem to subproblems that are individually solvable. And many of the cases, I would say 80% of the sub problems are deterministic problems and you don't need to use AI for it, mate. And that's such an interesting point. I would love to have a longer conversation with you, maybe even for the podcast, just because I think the way like what I'm really seeing now is that we've been in this game for a long time, we've seen the trends come and go. But if I look at what we're being faced with now, the skills that people need, they were the skills that we had like 15, 20 years ago in our shit hot developers and our analysts used to be part of teams to break down the problem and say actually who's doing what and when? Who do we want to do what and when? And how do we kind of break that down? Because you're right, if 80% of it is deterministic, we can automate that really easily nowadays and we can then offload. We say, do we want a human in the loop to deal with that piece or do we farm out to a model? Exactly. But that is a. They are skills which I wonder how prevalent they are now in the world. And there is this gap between this idea that builders can just go and build stuff when actually they can do that. But a lot of them might be utter shit because they haven't got requisite maybe last question what skills do you think are the most important to have? If someone's thinking about token economics and what they're doing, what should they be looking to build capabilities? So. Fortunately I always say the same thing. I think two things. One is strategy, and by strategy I really mean see the big picture and figure out that you don't start to over optimize too early. And the other is experimentation. I think none of the AI problems can be solved in a single shot. It's always going to be, let's try these 10 different approaches and okay, that one works and that one is cheaper and so on. Yeah, and I think experimentation, I will add on that. I think it's the ability to logically break down the problem and understand things. I've just spent the last weekend automating the podcast production, but because I know the process, it was easy for me to break it down. You're absolutely right. I make one call to a large language model in what is like a 20, 30 step process. Because the rest of it is just easy. Yeah, it's just there is a tool that already exists, you can call it, it's deterministic, it's easy to host, easy to run. And I think what I will say to everyone listening, if you do want to play around with this stuff, you can get a lot of stuff for free. So Google Cloud gives you a chunk of compute just to then go and use. There are large models out there for like Transcription, which gives like 170 hours of free transcription. You can buy credits for next to nothing and use up little bits of them. So if you're interested in experimenting, don't spend money, just go online, go to some forums, look at Reddit and find where you can get the free stuff. Because there's so much free stuff you can experiment with. Yeah, it's very likely that you're not gonna spend more than $100 in your experimentation. If you spend more then you spend too much. I think I spent on this workflow that I've done, which I've been running a lot. I think I've spent about 10 cents. Yeah, yeah, yeah, exactly. It is absolutely minimal. But I've learned so much. But if I hadn't spent out 10 cents, it would just be a paper based activity. And it reminds me many years ago of talking of design and there was a fake conference created called the Waterfall Conference. And one of its fake talks at the fake conference, sounds a bit weird, was about how the problem with designs isn't the design, it's the implementers who fuck it up and say if we can just keep our designs pure, then everything's okay. But the joke is that we know that no design stands up to that implementation, and we need to experiment and learn. It's always the hard part when you get your hands dirty, that's when you see the problems. Exactly. Because perfect is enemy of good. Tamash, it's been so nice to talk to you again. I always love seeing you. And everyone, please do go to Bobcat's coding, check out some of the papers they have on there, particularly on the token economy. It's available for just the cost of your email address, and there's no bad thing being signed up because you do produce some great content. So thank you very much, Tamash. Thank you for having me. Thank you, everyone, for listening. And we're back again with another talking 10 soon.