A new online course has been published in Classroom. It is course 118. Its title is "Kernel Density Estimation". Kernel Density Estimation (KDE) is a highly effective statistical method used in the energy sector. It allows you to take an existing dataset and generate new, realistic values that follow the exact same underlying patterns. This is perfect for when you need to simulate multiple scenarios. Specifically, in this course: - we look at a smart building with uncertain electricity demand, using 8760 hourly values (one year of data). - we want to simulate 1000 unique days where the demand is different but strictly follows the "logic" of our original dataset. - we walk through generating KDE-based data and using it to solve Monte Carlo and two-stage stochastic optimization models. These methods are absolute standards in the energy sector. Best of all, this is a highly applied course. I show you exactly how I used these exact techniques in a real-world energy project, so you can move past academic textbook exercises and start applying this to actual problems. The attached screenshots show the step-by-step process of how KDE is applied in industry. And also the differences between using KDE and non-KDE approaches ; KDE is more realistic. Non-KDE approaches are easier to model but lack realism.

In this course, we are developing a model in Python that uses Reinforcement Learning to minimize the operating costs of a Smart Building (also known as an 'energy hub'). The course is In the 'Classroom' at the very end (numbered 117). The full Python code is available for download. Definitely watch the videos as they explain all details necessary. Below is an explanation (using simple language with all jargon explained). Also you can download a slide in PDF (the slides in PowerPoint format are inside the course). I believe the best way to truly understand Machine Learning and Reinforcement Learning is by diving straight into the code. The code in these videos is based on a real-world application where I tested various methodologies to determine the most cost-effective way to operate a smart building. This building is 'smart' because of how it manages energy. It is equipped with its own energy storage unit (a battery), a solar photovoltaic (PV) unit, and a central computer system. Using advanced algorithms, this system autonomously decides: - When and how much to charge or discharge the battery. - Whether to curtail (reject) excess energy output from the solar PV unit. - When and how much electricity to purchase from the main grid. Over time, we have explored different algorithms for this central computer system. We covered Monte Carlo methods in Course 115 and Stochastic Optimization in Course 116. This new course (Course 117) focuses specifically on Reinforcement Learning. Reinforcement Learning is a subfield of Machine Learning. While traditional Machine Learning models learn by studying historical datasets, Reinforcement Learning trains an algorithm to learn through trial and error—interacting with an environment to figure out the best sequence of actions to achieve a goal. Machine Learning is a subfield of AI, and Reinforcement Learning is a specific branch of Machine Learning. Machine Learning models learn by studying datasets. It's called Reinforcement Learning similar to behavioral psycholog which says that we are learning through rewards and penalties. When the AI (the "agent") takes an action that gets it closer to its goal (in our case: saving money in the smart building) it receives a mathematical "reward." This positive feedback reinforces that specific behavior of the algorithm. Reinforcement Learning trains an algorithm (which is known as Agent) to learn through trial and error.

The attached slide shows a high-level summary of 2-stage and 3-stage stochastic optimization. A new 65-minute video has been uploaded in the online course 116 (in the classroom) and the full Python code for a 3-stage stochastic optimization model, exactly as it is used in practice, has been uploaded. In stochastic optimization, a stage is a point in time where you are forced to make a choice based only on the information you currently have. The future is uncertain. On the left side of the slide, you see the most common framework used in operations research. It assumes uncertainty is revealed all at once in a single massive dump of information in the 2nd stage. - Stage 1 (The "Here-and-Now"): The blue box in the attached slide. You must make a blind commitment before the uncertainty happens. (e.g., Bidding into a day-ahead electricity market at midnight). - Uncertainty Resolution: The sun comes up, and the exact weather and electricity demand are fully revealed. - Stage 2 (The "Recourse"): Now that you know exactly what happened, you take "wait-and-see" corrective actions to fix any imbalances. (e.g., Firing up a fast gas generator or discharging a battery to cover a solar shortfall). - The scenario tree: One root, splitting directly into final scenarios. On the right side, you see a more realistic model where information unfolds gradually over time, rather than all at once: - Stage 1 (Initial Decision): The blue box. The first six hours of the day. - First Resolution of uncertainty: You get a partial update. You now know the general weather for the day, but not the exact minute-by-minute fluctuations. - Stage 2 (Intermediate Decisions): The orange boxes. You make mid-course corrections based on this partial knowledge (our 10 branches from Hours 6-15). - Second Resolution: The evening arrives, and all remaining uncertainty is cleared up. - Stage 3 (Final Recourse): You make your final, fine-tuned adjustments - The Tree: This creates a deep scenario tree with multiple branching points.

I have found the manual for the solver 'CPLEX' , which is an alternative to 'Gurobi'. Both solvers are considered the best in Optimization and they are very popular. Using one of the two makes little difference. Both are fantastic. In practice, both are top tier commercial solvers for LP, MILP, QP, MIQP, and related classes. This is an AMPL focused quickstart guide that shows how to install and run CPLEX from Python via AMPL’s Python package (AMPLpy). I am sharing the CPLEX solver manual that I found.

Inside the 'Classroom' , there is a new course (116) , which shows how to develop, in Python, a 2-stage and a 3-stage stochastic optimization model. The code is available for download, and it is explained through a video of about 1 hour in total. The prerequisites are courses 115 (deterministic optimization) and 116 (Monte Carlo). Stochastic Optimization is used a lot in energy, economics and finance. Anytime we have something uncertain, we use scenarios to describe how the future may play out. For example, the electricity demand tomorrow can be 100 kW, 50kW, 20kW. So we have 3 scenarios. We can have as many scenarios as we think is reasonable. For example in the code we build a 'scenario tree' consisting of 1000 scenarios. And we then assign a probability to each of these scenarios. Then, we have an objective function, which includes probabilities. We have constraints. And this is like any other optimization model. We call it 'stochastic' because it has probabilities in the objective function and because it has scenarios. It is like any other optimization model. So , the jargon may sound a bit intimidating , but it is very simple actually. We apply stochastic optimization to a smart building, which has a solar Photovoltaics unit and also it has residents who consume electricity (electricity demand). And we want to minimize the daily cost of operating this smart building in the future. Since we want to 'minimize' something' we speak about 'optimization'. And also we have uncertainties: the electricity demand is uncertain. Also, the output of the solar PV unit is uncertain. See the two screenshots attached for some extra context.