Transfer Learning for Solar Energy Production Forecasting with LSTM: Generalized vs. Specialized Models

Question

I am working on a solar energy production forecasting problem using LSTM multi-step models to predict 1/4/8h ahead of solar energy production for different solar installations. Our goal is to help clients optimize their energy utilization by trading with their neighbours or respective Microgrids.

I have clustered households into groups such as small generators, medium generators, and large generators. I am currently developing a multi-household model for each cluster using TensorFlow's LSTM multi-step model tutorial.

To improve prediction accuracy and provide a more personalized approach, I would like to explore transfer learning to create specialized single-household models based on the generalized multi-household models.

Multi-Household Model (Generalized Model)

The dataset consists of 160 time series and includes weather features such as hour, day, month, temperature, DHI, DNI, GHI, precipitation, and solar zenith angle. The model learns from multiple similar households.

To better visualize the dataset, here is an example:

Hour	Day	Month	TS_0	TS_1	TS_N	Temperature	DHI	DNI	GHI	Cosine Periodicity	Sin Periodicity	Other Features
6	1	5	0	0	0	15	…	…	…	…	…	…
7	1	5	0.1	0.1	0.1	17	…	…	…	…	…	…
8	1	5	0.2	0.3	0.25	18	…	…	…	…	…	…
9	1	5	0.5	0.4	0.35	18	…	…	…	…	…	…
10	1	5	1	0.8	0.85	20	…	…	…	…	…	…

Note: These features related to the weather would be an average of the district that these houses exist in.

This current setup utilizes TS_0 to TS_N as examples to learn from each other since their solar installations are similar and should therefore yield similar amounts of electricity. I can then use TS_X as an output label to predict, thereby getting a prediction for each household while maintaining some learning from other household examples.

Single-Household Model (Specialized Model)

I want to create a specialized model for each household by using transfer learning from the generalized model. This specialized model will incorporate household-specific features such as solar capacity, number of habitants, solar installation angle/direction, and town-specific weather parameters.

Objective

The goal is to create generalized models that can help train specialized models for better accuracy for each household while allowing easy onboarding of new users.

Business Case for Generalized vs Specific Model

The generalized model would be trained for general locations across a country for certain solar installations (this would be trained with ongoing customer data). The more specific single-household model would enable (in theory) to have more personalized predictions for your specific solar installation setup and location.

A generalized model approach would enable the solar installation company to be able to onboard users more easily - simply add a new customer to the generalized model that better fits a specific household cluster. A single-household model would make it more difficult to onboard a new user as you would need a buffer period to gather customer data before being able to train a specific ML model for them.

Problem

The problem with the single-household model is that the household-specific features might be constant across the dataset and not very meaningful. Adding these features in the multi-household model would lead to a high-dimensionality problem where we would have a feature for each timeseries.

I would like to ask for recommendations on the following:

1. How can I create an architecture that goes from generalized models to more specific models while being able to introduce valuable additional information specific to a single household?
2. If I opt for a multi-household generalized model solution, how can I include more specific feature information for each time series without running into dimensionality problems?
3. If I choose a single-household model solution (a model for every single client), how can I ensure good predictions, considering the model wouldn't have access to other time series examples within its tier?
   1. There is a limit to the amount of single-household data we could acquire since some of these may be more recent customers and won’t have multi-year data available

Answer 1

I would probably refrain from using transfer learning. Transfer learning works if you have a more specific data set. Transfer learning on a single sample is going to be very hard, and I'm unfamiliar with any papers that attempt such a feat.

Model conditioning is when you add variables as 'condition' to the rest of the input of your model. In your case, you have several general variables (such as weather?). During general training, you can sample from your dataset (do some feature engineering or such) to create your conditional variables. Then upon inference, you can get your specific condition, together with the general variables, input them through your model and get an output for your specific case.