In this course, we are developing a model in Python that uses Reinforcement Learning to minimize the operating costs of a Smart Building (also known as an 'energy hub'). The course is In the 'Classroom' at the very end (numbered 117). The full Python code is available for download. Definitely watch the videos as they explain all details necessary. Below is an explanation (using simple language with all jargon explained). Also you can download a slide in PDF (the slides in PowerPoint format are inside the course). I believe the best way to truly understand Machine Learning and Reinforcement Learning is by diving straight into the code. The code in these videos is based on a real-world application where I tested various methodologies to determine the most cost-effective way to operate a smart building. This building is 'smart' because of how it manages energy. It is equipped with its own energy storage unit (a battery), a solar photovoltaic (PV) unit, and a central computer system. Using advanced algorithms, this system autonomously decides: - When and how much to charge or discharge the battery. - Whether to curtail (reject) excess energy output from the solar PV unit. - When and how much electricity to purchase from the main grid. Over time, we have explored different algorithms for this central computer system. We covered Monte Carlo methods in Course 115 and Stochastic Optimization in Course 116. This new course (Course 117) focuses specifically on Reinforcement Learning. Reinforcement Learning is a subfield of Machine Learning. While traditional Machine Learning models learn by studying historical datasets, Reinforcement Learning trains an algorithm to learn through trial and error—interacting with an environment to figure out the best sequence of actions to achieve a goal. Machine Learning is a subfield of AI, and Reinforcement Learning is a specific branch of Machine Learning. Machine Learning models learn by studying datasets. It's called Reinforcement Learning similar to behavioral psycholog which says that we are learning through rewards and penalties. When the AI (the "agent") takes an action that gets it closer to its goal (in our case: saving money in the smart building) it receives a mathematical "reward." This positive feedback reinforces that specific behavior of the algorithm. Reinforcement Learning trains an algorithm (which is known as Agent) to learn through trial and error.

The attached slide shows a high-level summary of 2-stage and 3-stage stochastic optimization. A new 65-minute video has been uploaded in the online course 116 (in the classroom) and the full Python code for a 3-stage stochastic optimization model, exactly as it is used in practice, has been uploaded. In stochastic optimization, a stage is a point in time where you are forced to make a choice based only on the information you currently have. The future is uncertain. On the left side of the slide, you see the most common framework used in operations research. It assumes uncertainty is revealed all at once in a single massive dump of information in the 2nd stage. - Stage 1 (The "Here-and-Now"): The blue box in the attached slide. You must make a blind commitment before the uncertainty happens. (e.g., Bidding into a day-ahead electricity market at midnight). - Uncertainty Resolution: The sun comes up, and the exact weather and electricity demand are fully revealed. - Stage 2 (The "Recourse"): Now that you know exactly what happened, you take "wait-and-see" corrective actions to fix any imbalances. (e.g., Firing up a fast gas generator or discharging a battery to cover a solar shortfall). - The scenario tree: One root, splitting directly into final scenarios. On the right side, you see a more realistic model where information unfolds gradually over time, rather than all at once: - Stage 1 (Initial Decision): The blue box. The first six hours of the day. - First Resolution of uncertainty: You get a partial update. You now know the general weather for the day, but not the exact minute-by-minute fluctuations. - Stage 2 (Intermediate Decisions): The orange boxes. You make mid-course corrections based on this partial knowledge (our 10 branches from Hours 6-15). - Second Resolution: The evening arrives, and all remaining uncertainty is cleared up. - Stage 3 (Final Recourse): You make your final, fine-tuned adjustments - The Tree: This creates a deep scenario tree with multiple branching points.

Would like to hear your thoughts on the potential global economic consequences of the closure of the Strait of Hormuz? I have some takeaways about this situation: 1.The closure of the Strait of Hormuz and halted shipments could push Brent crude prices above USD 100/bbl if the blockade persists. 2.Geopolitical tensions and market uncertainty typically increase demand for safe-haven assets like gold. There are increasing doubts about the US dollar's safe-haven status from Central banks like the European Central Bank, this could suggest a potential shift in currency dynamics making other currencies (Swiss Franc) or commodities ( Gold or Silver) more appealing as a safe heaven. I believe we will see the same market drivers for 2026 specially that war is escalating very fast, we are seeing even more countries involved like many of the Gulf countries, . Dr.Spyros made a analysis of Gold rally in 2025 (It explains many of the market drivers we are seeing also in 2026, highly recommended!) 3. Energy Resilience The uncertainty surrounding the closure of the Strait of Hormuz raises important questions about the resilience of global energy systems. Given that there is no clear timeline for reopening, this disruption could act as a catalyst for countries to reassess their dependence on oil and gas. Several countries are already feeling the effects of supply constraints, and if the situation persists, we may see increased volatility in energy markets. In this context, the crisis could accelerate investment and policy shifts toward renewable energy sources, as nations seek to reduce exposure to geopolitical risks associated with fossil fuel supply routes. If the conflict evolves into a prolonged situation, repeated disruptions or closures over time could further reinforce the urgency of diversifying energy portfolios and strengthening energy security. Let me know your thoughts!

I have found the manual for the solver 'CPLEX' , which is an alternative to 'Gurobi'. Both solvers are considered the best in Optimization and they are very popular. Using one of the two makes little difference. Both are fantastic. In practice, both are top tier commercial solvers for LP, MILP, QP, MIQP, and related classes. This is an AMPL focused quickstart guide that shows how to install and run CPLEX from Python via AMPL’s Python package (AMPLpy). I am sharing the CPLEX solver manual that I found.

Recent interview question in Chevron. Happy to have some inputs. Thanks . - Department: Corporate Strategic Planning, interviewing jointly with Chevron New Energies (the division responsible for lower-carbon investments like hydrogen and carbon capture). - Target Role: Senior Quantitative Energy Economist - Interview Stage: Final Round / Executive Panel Presentation. You would likely be standing at a whiteboard in front of 3 to 4 senior directors. - 2 February 2026. - Format: the recruitment coordinator takes your mobile phone, laptop, and smartwatch. There is no AI, no internet, and no Python to run your simulations. You are led into a quiet focus room. On the desk is a printed piece of paper containing "The Carbon vs. Capital Conundrum" prompt, a basic scientific calculator, a notepad, and a pen. You are given exactly 45 minutes to digest the prompt, formulate your economic models from memory, and structure your recommendation.After 45 minutes, you are escorted into the boardroom to face the senior directors. You have nothing but your handwritten notes, a whiteboard, and a marker. Interview Question: " You are presenting to our executive investment committee. We have a strict capital expenditure (CapEx) limit for the upcoming fiscal year and can only fully fund one of two mega-projects. You must recommend which one we choose: Project Alpha (Deepwater Oil & Gas) - Location: Offshore West Africa - Financials: Spectacular projected Internal Rate of Return (IRR) of 20% with a very fast payback period. - Risks & Downsides: The host country is experiencing growing political instability. Furthermore, the project has a massive carbon footprint that will push our corporate emissions well over our stated public reduction targets for the decade. Project Beta (Carbon Capture & Hydrogen Hub) - Location: US Gulf Coast - Financials: The economics are extremely tight. The baseline IRR is only 7%, which barely clears our corporate hurdle rate (minimum acceptable return). - Benefits: It operates in a highly stable geopolitical region, secures massive government tax credits, and practically guarantees we hit our corporate net-zero pledges.