Token counter · AI Automation Society

Token counter

Hello everyone! I need to find a way to count how many tokens a text would consume for a specific model. I'm creating an assistant in n8n that summarizes the demo calls I have with my leads, but the calls are often very long and exceed the maximum token limit that can be sent. Is there any way to count how many tokens the transcript text will consume?

I found a Python library called tiktoken that is able to do this but I needs to be installed. I don´t know if it´s possible to do that.

Thanks!!

2 comments