I've been experimenting with Langfuse, Open Source LLM Engineering Platform, and I think it's a very good solution that let's you have logging and management locally, with a very small footprint. It's really simple to deploy locally, https://langfuse.com/docs/deployment/local . I wanted to add another way, for those that are looking to possibly do group collaboration that could lead to some financial incentive(this part i'm still figuring out). Here's a simple outline of how to deploy this on AWS, which will cost about $32-33/month. ## Creating RDS instance
Navigate to the RDS service in the AWS console
Create Database
- Change the Engine to PostgreSQL
- Select `Free tier` from the Templates section
- Change `Master username` and `Master Password` to something acceptable to your organization
- Change `Storage type` under `Storage` to `General Purpose SSD(gp3)` make sure the allocated storage is 20 GiB
- Under `Connectivity` change `Public access` to `Yes`
- Under `Connectivity` change `VPC security group (firewall)` to `Create new` then give it a name
- Scroll down and click `Create database`
Now, we want to create our EC2 instance.
- Assuming you know how to create an EC2 instance. The noteworthy setup here is using Ubuntu 22.04 on a free tier instance.
- Create a new Security group, take note of the name, we're going to use it in the next step.
- Note the public IP, this is going to be used to setup your langfuse instance.
Next we modify the security group for the RDS instance
- Navigate to RDS in the console
- Select your DB from the Databases menu in RDS
- With the `Connectivity & security` tab selected, click the link under `VPC security groups`, this should be the name you created for your security group in the previous steps.
- Click the link under the column `Security group ID`
- Click `Edit inbound rules`
- Click `Add rule` select `PostgreSQL` from the `Type` dropdown. Select `Custom` from the Source dropdown. Click the box next to the source dropdown, type in the name of the security group you used for your EC2 instance.
- Click `Save rules`
Now you can install Docker and Langfuse on your EC2 instance
## Installing Docker new EC2 - Ubuntu 22.04
```bash
sudo snap install docker
docker run --name langfuse \
-e DATABASE_URL='postgresql://postgres_username:postgress_password!@aws_rds_endpoint:5432/postgres' \
-e NEXTAUTH_SECRET=some_secret_you_create \
-e SALT=some_salt_you_create \
-p 3000:3000 \
-a STDOUT \
```
Navigate to your publicIpAddress:3000 and you should be able to utilize Langfuse in a semi private setting.
Happy LLMing!