M2 internship: Optimised energy management of a fleet of electric vehicles in a large-scale smart grid based on multi-agents multi-armed bandits

Optimised energy management of a fleet of electric vehicles in a large-scale smart grid based on multi-agents multi-armed bandits

Supervisors

- Anne Blavette, CNRS Permanent researcher, IETR Lab, ENS Rennes, France (anne.blavette@ens-rennes.fr)

- Raphaël Féraud, Researcher, Orange Labs, Lannion, France (raphael.feraud@orange.com) 

- Patrick Maillé, Professor, IRISA lab, IMT Atlantique, Rennes, France (patrick.maille@imt-atlantique.fr) 

- Guy Camilleri, Assistant professor, IRIT lab, Univ. Paul Sabatier, Toulouse, France (Guy.Camilleri@irit.fr)

- Hamid Ben Ahmed, Professor, IETR Lab, ENS Rennes, France (benahmed@ens-rennes.fr)

Topic

In order to integrate more and more renewable energy, the power system is being transformed into a smart grid. However, this transition will require shifting from a centralized management of flexible entities (power sources (e.g. PV, etc.), electric vehicles, …) to a highly decentralized, smart and dynamic energy management. It will also require considering a large number of flexible entities, multiple sources of uncertainties, constraints in the electrical network, etc. This represents a very complex problem that conventional methods are not able to address. In this context, there is a clear need for research works contributing actively to the development of decentralised energy management strategies for large-scale smart grids under uncertainty.

Traditional methods may tackle only small-scale problems (tens of flexible entities) as a greater number would lead to an explosion of the required computing time. In the search for relevant methods to tackle this problem, several approaches are being investigated. Among them, methods based on multi-argent multi-armed bandits may present high scalability capabilities which render them particularly suitable for the real-time operational management of large-scale smart grids [Rizk2018],[Sutton, 2018]. In this perspective, research works are already being conducted in collaboration between IETR, IRIT, IRISA and Orange Labs on the problem considered here, and for which promising results have already been obtained [Zafar2023].

The goal of this internship is to build upon this research work, and expand an already developed methodology for the decentralised management of flexible entities (e.g. electric vehicles) based on recent advances in the field [Féraud2019]. In the proposed internship, a methodology will be developed for integrating the dynamic power flow limitations imposed by the transmission system operator (TSO) at the interface between the TSO network (i.e. high voltage) and the network managed by the distribution system operator (DSO) at medium/low-voltage. These limitations may be satisfied among others by adapting the consumption/production of flexible entities (e.g. stationary batteries, electric vehicles) connected on the DSO network. Hence, the time-varying constraints imposed by the TSO will therefore result in dynamic congestion levels as opposed to fixed congestion levels as applied nowadays (and currently considered in the methodology developed in our previous/current work). The expansion of the previous/current work will also consider different bandits algorithms.This internship will be carried out in the frame of the national TASTING project, in collaboration with transmission system operator RTE.

Tasks description

The position will include the following non-exhaustive list of tasks:

-       Bibliographical search in the scientific literature

-       Mathematical formalization of the considered scientific problems

-       Numerical simulations (code development, testing and validation, experimentation, results discussion)

-       Regular reporting to the supervising team

-       Scientific publication writing

Skills

Student in Master 2 at a University, an Engineering School or equivalent, in the field of machine learning, computer science, applied mathematics, or electrical engineering with a strong multi-disciplinary background. 

Excellent programming skills in Python (including knowledge of numpy library) is also required. Knowledge in reinforcement learning would be greatly appreciated, but applicants with no knowledge and a strong motivation and interest for reinforcement learning will be considered (training can be provided during the internship). Knowledge in multi-armed bandits would be a plus.

A strong capability to work in a team and communicate within a multidisciplinary team, both onsite and at a distance, would be very appreciated.

Knowledge and/or an interest in renewable energy, smart electrical networks would be a plus.

Application 

Please send your CV and cover letter to all the supervisors listed at the beginning of the document.

Starting date

Flexible starting date from February 2025 (but any later starting date is possible).

Stipend

Around 700€/month (negotiable depending on skills). In addition, the student may be eligible to housing allowance (https://www.adele.org/en/housing-aids), as well as a room in the ENS student residence (https://www.espacil-habitat.fr/devenir-locataire/etudiants/residences-etudiantes/residence-rose-dieng-kuntz-rennes/).

Location

The internship will be carried out mainly at ENS Rennes.

References

 [Féraud2019] R Féraud, R Alami, R Laroche, Decentralized exploration in multi-armed bandits, ICML 2019.

 [Rizk2018]  “Decision Making in Multiagent Systems: A Survey”, Y. Rizk, M. Awad, E. Tunstel, IEEE Trans. on Cognitive and Developmental Systems, vol. 10, no. 3, pp. 514-529, Sept. 2018. 

[Sutton2018]  Richard. Sutton and Andrew G. Barto, “Reinforcement Learning: An Introduction”, 2nd edition, MIT Press, Cambridge, 2018.

[Zafar2023] Sharyal Zafar, Raphaël Féraud, Anne Blavette, Guy Camilleri, Hamid Ben Ahmed, Decentralized Smart Charging of Large-Scale EVs using Adaptive Multi-Agent Multi-Armed Bandits, in Proc. 27th International Conference & Exhibition on Electricity Distribution (CIRED 2023), Rome, Italy.