Ujwal Dinesha

TL;DR. Most alignment pipelines stack two fragile pieces — fit a reward model on noisy preference data, then fine-tune the LLM against it. PITA throws both out. We learn a small guidance policy that nudges the base model’s next-token distribution at inference time, directly from preferences, with zero LLM fine-tuning.

The trick is to frame alignment as identifying a latent preference distribution and solve it with stochastic search. The guidance model produces exponentially-weighted Q-values that re-shape the LLM’s logits on the fly. The result: a much cheaper alignment recipe that side-steps reward-model instability.

We test PITA on math reasoning, TL;DR summarization, and sentiment control. The base LLM stays frozen the whole time — you can swap it out, swap the guidance policy in, and keep going.

TL;DR. Restless multi-armed bandits (RMABs) usually assume the decision-maker observes a clean scalar reward when they pull an arm. In the real world — healthcare, online ads, app engagement — you almost never get that. You get which option did the user prefer. DOPL is the first online algorithm for RMABs in this setting with provable regret.

We introduce Pref-RMAB, where the planner sees only pairwise preference feedback between activated arms. DOPL interleaves exploration of the unknown transition kernels with preference-data collection, then plans directly against preferences instead of trying to reconstruct rewards. The analysis lands at Õ(√(T · ln T)) regret.

Practically, we show it works on three benchmark domains — a CPAP adherence model, the Armman maternal-health setup, and an app-marketing engagement environment. Codebase is built around a config-driven runner so it’s easy to plug in your own RMAB environment.

TL;DR. Standard RLHF maximizes expected reward, so it’s perfectly happy with a model that’s great on average but occasionally spits out something toxic. RA-RLHF swaps the objective for Conditional Value at Risk — the average reward of the worst-α tail — which directly punishes those rare bad generations.

The fix is surprisingly clean: at each PPO step, we sort the batch by reward and weight the tail more aggressively. No new architecture, no new reward model, drop-in over TRL.

On IMDB sentiment-control and Jigsaw toxicity-mitigation benchmarks, RA-RLHF cuts the rate of toxic / off-target outputs without giving up on the generation quality you’d get from vanilla RLHF.

TL;DR. A base station is streaming video to N devices over a shared wireless link with a tight energy budget. Each device’s buffer / channel state arrives back at the BS with a deterministic delay. That delay is the whole problem — the controller is acting on stale information.

We cast this as a cooperative multi-agent constrained POMDP and use strong duality to split it into N independent transmitter-receiver pairs that all share a single Lagrange multiplier (the energy price). Each pair is then solved using the common-information approach with approximate information states.

The neural architecture that drops out of this analysis is delay-aware by construction and beats transmitter-only baselines in simulation. It’s a nice example of how a careful POMDP decomposition can give you tractable learning where the naive joint problem would be hopeless.

TL;DR. Open RAN’s near-real-time RIC sits in the cloud and takes > 15 ms to decide anything. That’s far too slow for per-TTI scheduling in a real cell. EdgeRIC pushes the RIC down to the edge, co-located with the DU, and gets the decision loop under 1 ms.

The system exposes a real-time E2 interface (we call it RT-E2) built on ZMQ + protobuf, with TTI-level synchronization and per-UE KPI reports. Above that we run μApps — small policies, including a PPO-trained MAC scheduler — that can react fast enough to actually matter.

End-to-end, the AI-driven scheduler beats classical heuristics (max-CQI, proportional fairness, round-robin) by 5–25% on throughput and downstream application metrics, all on real srsRAN hardware.