AI for Decision Making

The "Construct" research project developed software tools to study small group behavior using multiplayer videogames and simulations. They also enabled machine learning systems to recognize complex human behavior as it emerges within these environments.

Decision Making

Scientific Visualization

Representing Data

The research uses a collaborative, goal-oriented multiplayer videogame environment to render data about game status, game environment, player sensory data, player behavior and player communication, in real time, to a database and algorithms that then seek to derive "meaningful abstractions".

These findings are rendered in an elegant timeline-oriented software tool that seeks to reveal significant game milestones in each data dimension. The system also supports manual annotation of periods of play as being representative of one or more behavioral states. The architecture then interacts with an established machine learning mechanism to automatically recognize complex group behaviors in subsequent play.

Scientific Visualization

Arrow's Contribution

Development Process

Arrow Digital worked closely with researchers at Columbia College on the development and architecture of the project. We architected and developed the portions around the data, communication, data abstraction and visualization for researchers. Arrow developed the communication protocol for simulation and game engines to send action log information to the database in an abstracted way which would allow for different game engines and platforms.

Development Process

The Outcome

A Simulation Protocol

Using advanced SQL based procedures to facilitate abstractions we were able to deliver data that was meaningful to machine learning engines and researchers through a live and after action review tool. We pushed the envelope with new technologies like Windows Presentation Foundation, LINQ and XAML. The vector based UI of WPF allowed us to bring features like timeline interval smooth zooming, custom graphing and video review simulation to the project. These features gave researchers a powerful view into the data collected from the simulations and serious games.

Additional Features

  • Play back ability
  • System interactions and incoming data serves as teaching mechanisms
  • Users can save and share layout
  • User can add additional visualizations & columns
  • Annotation of what was said Visualization of transcript
  • Categorized team behavior

Result

The toolset was used by researchers at Columbia College and DePaul University.

*Results are classified.