Menu

What we study

Reliable AI under conflict, uncertainty, and feedback

Powerful models often look reliable in clean, single-turn tests and fail when objectives conflict, evaluation signals are biased, tools carry different costs, or an earlier output must be repaired. Our research turns these pressure points into measurable questions and uses controlled experiments to understand both observable behavior and the mechanisms behind it.

Research Areas

Four directions

Each area asks where a capability that looks solid in aggregate breaks down under a more demanding setting.

  • Trustworthy Evaluation

    We study bias, robustness, uncertainty, and faithfulness when language and vision–language models are used as evaluators.

    • LLM-as-a-Judge
    • Evaluation
    • Robustness

  • Reliable Agents

    We examine instruction hierarchy, tool use, cost-aware decisions, and safety when agents face conflicting or misleading signals.

    • Agent Reliability
    • Tool Use
    • AI Safety

  • Feedback & Multimodal Systems

    We evaluate whether models can diagnose failures, express actionable feedback, and carry out precise corrections without causing new errors.

    • Multimodal AI
    • Feedback
    • Image Editing

  • Data-Centric & Korean NLP

    We develop evaluation and training data, metrics, and analyses that reflect language-specific and real-world model behavior.

    • Data Generation
    • Evaluation Data
    • Korean NLP

Ongoing Projects

What we are working on now

Three directions currently drive most of the experimental work.

Ongoing

Cost-Aware Tool Use in LLM Agents

  • LLM Agents
  • Tool Use
  • Cost-Aware Reasoning

As agents take on more work through external tools and APIs, choosing a tool stops being a purely functional decision. Several tools may accomplish the same thing at very different cost, and an agent running unattended makes that call many times over. Deployed systems are assumed to weigh capability against cost sensibly, and there is little systematic evidence that they do.

We study how reliably agents make that trade-off, and which properties of how a tool is presented pull the decision away from the sensible option. This is a failure mode that rarely shows up as a wrong answer — it shows up as a system that is quietly more expensive and less predictable than it should be, which is exactly the kind of problem that survives into production.

Ongoing

Instruction Hierarchy Under Accumulated Conflict

  • AI Safety
  • Instruction Following
  • Model Behavior

A deployed language model takes instructions from several places at once: a system prompt, a developer, a user, retrieved documents, the output of tools it called itself. Much of its safety rests on respecting the order of authority between those sources. A single clean conflict is the easy case, and models mostly handle it.

Real deployments are not the easy case. Instructions accumulate, contradict each other, and arrive from sources of very different trust. We study how instruction priority holds up under that pressure and why it gives way, because a system that follows the right source only when the situation is simple is not one that can be relied on where it matters.

Ongoing

Feedback and Correction in Image Editing

  • Multimodal Evaluation
  • Image Editing
  • Feedback

Most work on instruction-guided image editing measures whether a model carries out a fresh instruction. Far less is known about what happens once an attempt has already gone wrong — which is the situation people actually run into. Real editing is iterative: a user asks for something, the result is close but not right, and the exchange continues from there.

We work on evaluating how multimodal systems handle that case. The questions we care about are whether a model can tell that a result missed the intent, and whether it can correct the result without damaging everything else in the image. Both matter for editing tools people can rely on, and neither is captured by measuring success on a single pass.

Methodology

How we work

We combine benchmark design, behavioral evaluation, human and model-based analysis, and controlled experiments. Across projects, we aim to separate capabilities that are usually measured together, so that an evaluation tells us not just whether a system failed, but which ability failed and under what conditions.

  • Measure carefully

    Design evaluations that isolate the capability we actually want to understand.

  • Stress realistic systems

    Study behavior under conflict, feedback, imperfect outputs, and deployment constraints.

  • Explain failures

    Move beyond aggregate scores to understand why a system behaves the way it does.

Collaboration

Work with us

We are open to academic and industry collaboration on evaluation, agent reliability, and AI safety, and to students who want to take on these questions with us.

Research opportunitiesyerinhwang@dongguk.edu