How should I include constraints on the state variables

[lydiarong]

In the cartpole scenario,there are no constraints in the state space,but I want to control the position of the cart between ±0.2m,I manually add the constraints in predict_policy function of system.hpp as shown below ,it works well when the constraints is ±2.5m,but when it comes to ±0.2m,it fails.

[costashatz]

In the cartpole scenario,there are no constraints in the state space,but I want to control the position of the cart between ±0.2m,I manually add the constraints in predict_policy function of system.hpp as shown below ,it works well when the constraints is ±2.5m,but when it comes to ±0.2m,it fails.

This is not the proper way of inserting the constraints.. Any system behavior should be described in the system function. As cartpole is modeled as an ODE, this means that you need to alter the dynamics function.

Nevertheless, I have a feeling that 0.2m is too small and thus there is no space left for the cart to make the movements that would allow the pole to swing-up.

[lydiarong]

Thanks for your reply,I'll try this.But if I want to apply this algorithm to a real physical model,then it's useless to alter the dynamics function even it can work in simulation,right? How should I add the state constraints when it comes to a real physical cartpole model?

[costashatz]

But if I want to apply this algorithm to a real physical model,then it's useless to alter the dynamics function even it can work in simulation,right? How should I add the state constraints when it comes to a real physical cartpole model?

In a physical cartpole, the limits will be on the hardware system, right? Or in the controller implemented on the physical system, no? I mean there will be a physical stop or something in the controller that stops the cartpole (and this should not be given to the algorithm, except if you want to give some prior knowledge)..