Local vs cloud , Help me choose · AI Developer Accelerator

Local vs cloud , Help me choose

I need a new system cause mine is 8 y/o. I am working on learning programming, full stack, DS & ml. Over the course of next year & half. While I start my business.

I quit my job & plan to change lanes so I am working on savings. So my highest concern is economic efficiency.

I saw @tom-welsh-8986 post & got some clarity on running locally.

The issue is. I’m torn between 4 options

M3 max MacBook Pro with 64 gb ram. + cloud
Pro art laptop + cloud
Pc 1 = 14700 + 4090 + 128 gb ddr4. = ~$4,100
Pc 2 = threadripper 7960x + A6000 + 128gb ddr5 ~$8,10 .

Cloud compute - paperspace vs AWS vs vertex. = a rough estimate of 1000 - 2000 usd per year (minimum) ~5,000 - 7,000 usd per year.

( the cloud compute cost was calculated based on average havy api usage of individual with agents & a near accurate estimate of the tokes that need to be read & generated based on the context window & frequency)

I completely accept that this system is Going to be obsolete in 3 years max. And currently my savings would let me spend a maximum of 4K on compute ( pc/ cloud) I can borrow 4 more from my parents. That’s it. So… I can not even think of local servers.

Which would have the above would best suit the following use cases.

———————————————————

# Context & use case

Privacy is not a greatest concern. But it will have a lot of sensitive information.

I plan on running multiple agents simultaneously

& one of which will constantly loop through the following.

1. figure out the context & essence of my question, by predicting & refining what kind of answer i expect with a every question i post.

2. ask for clarification to ensure it's assumptions, align with reality

3. split the question i asked into multiple questions to ensure i get a comprehensive response.

5. rephrase the question following to get an answer that align with the requirements.

6. deterring the order / sequence to ask questions to generate the best answer.

7. generate multiple responses in separate threads to determine which aligns with the requirement the most.

8. monitor & record it's internal processes while generating the responses.

9. check each of the response for hallucinations & factual inconsistencies. using a RAG db or by searching the internet.

10. repeat, refine, iterate & give itself feedback on this process to improve this process.

so by the end of it, it slowly gets better at generating the Answers i need & not the one i want, or some generic word vomit ( I know I know that’s an insane & insanely complex thing )

It needs to gain inferences from my journals, integrate with my mobile applications ( phone, task manager, time tracker, screen time, app usage, health tracker, diet & fitness app…. )

I need to build a RAG db for my books, papers & questions…. To function as my learning assistant

8 comments