I am trying to create a set of ML models that will serve as a replacement for a complex deterministic simulation. The simulation requires 4 inputs (x1, x2, x3 and x4) to determine 4 different outputs (y1, y2, y3 and y4). x1, x2, x3, x4, y1, y2 and y4 are all floats, while y3 is an int. My constraints are (.5 <= x1 <= 10.), (313.3 <= x2 <= 317.5), (9.6 <= x3 <= 14.4), (1.01 <= x4 <= 27.6). As for interpretability, y1 is the output of a cost function, while the other outputs are a description of a system that yields y1. If you wanted to predict the cost of a car to use in a roadtrip based on the travelling distance, fuel, time spent on the road, etc as inputs, y1 would be the cost and y2, y3 and y4 could stand for the brand, horsepower and engine type for example.
I have decided to build 4 ML models for each of the simulation outputs. Since the simulation is deterministic there shouldn't be any random errors when evaluating an input set: the input [x1, x2, x3, x4] will always yield the same value y1, y2, and so on. I have compared and selected a few promising regression models using sklearn, but I am still unsure how to properly handle the preprocessing step of my pipeline.
I tested some of the available preprocessing steps (QuantileTransformer, PolynomialFeatures, ...) which seemed to increase the final score (r²) of the pipelines, but would they be adequate for a deterministic data set in particular? Wouldn't I be removing/altering important information from the data set/simulation process?
And besides, how would I know which preprocessing step is compatible with my data without having to test one by one?
Inputs x1 x2 x3 x4 0 8.06675 316.4437 13.6680 19.200474 1 4.93175 315.9649 11.6328 23.000562 2 8.45625 313.4827 12.7032 24.640326 3 2.32875 315.2509 10.2216 26.540370 4 2.88925 315.8053 10.3944 4.728906 Outputs y1 y2 y3 y4 0 1.101 320.353 2.0 228.620 1 1.119 327.815 2.0 170.150 2 1.091 327.280 2.0 304.725 3 1.142 331.856 2.0 94.460 4 1.316 318.390 1.0 24.080