Fitted Occupancy-Ratio Evaluation without Bellman Completeness

Occupancy ratios correct distribution shift in offline reinforcement learning and are central to off-policy evaluation.

Occupancy ratios correct distribution shift in offline reinforcement learning and are central to off-policy evaluation. Existing primal-dual and minimax methods typically estimate these ratios by enforcing occupancy-balance moments over a critic class. We propose fitted…