[AAMAS09] Multi-Agent Learning II. Emergent Behaviour II

Well, well, well, finally I’m in a room

Stigmergic Landmark Foraging
Nyree Lemmens, Karl Tuyls


Integrating Organizational Control into Multi-Agent Learning
Chongjie Zhang, Shereif Abdallah, Victor Lesser

Problems of distributed learning

  1. 1
  2. 2
  3. 3

Basic idea: organisational-based supervision framework. It’s a multilevel structure (recursive?) Lowest level network agents are ‘workers’. Each leaning agent reposrt its abstract state to its inmediate supervisor and them use rules a suggestions to transmit its supervisory informatio to its subordinates. Rules are set of forbidden actions and suggestions are actions with a degree in [-1,1], Rules are hard constraints and sugg are soft constraints that represent preferences.

The problem they’ve used to test this model is DTAP (distributed task allocation problem). Using a 27×27 agent grid… only!!!, too small!! I can manage several millions of agents to do the same :-( The results: interesting, but I don’t
understand all this stuff to be used in a small network as this: two supervision levels for such a group of agents.

It scales, but adding more supervision levels that may affect to the performance. I don’t like it. You’ll need a lot of layers for a really big network. Furthermore, in the experiments they’ve used a grid instead of a network and this is not ‘elegant’.

Multiagent Learning in Large Anonymous Games
Ian Kash, Eric Friedman, Joseph Halpern

We need to learn quickly, with minimal information and despite of noise. And to test their method they’re using games, but instead of being game theoretic games, they’re continuous, anonymous and designed games. He explains the method with a simple game but at the end it’s similar to game theory… I hate utility functions for agents. The behaviour can’t be reduced to a number or a function. Agents are more complex that that. We are more complex than that.

A simple algorithm to adapt the agent’s behaviour to the rest, so the dynamic converge despite of having agents making mistakes (so they’re introducing noise) in their decisions. As the number of agents increases, the system is more stable and converges faster… they’ve tried with 100 agents (again, too small for me). This results allows to tolerate strange behaviours.

Learning of Coordination
Francisco Melo, Manuela Veloso

Problem: many MAS solutions assume full joint sate observability because consider only local observability makes the problem too complex to be solved. But in many of these problems agent interactions are local. So they have to learn when interaction/coordination is advantageous. MDP and Q-learning is useed. And to show how it works, with an example of two robots that have to cross a gate.

They introduce a Coordination action (pseudo-action) and agents have to decide when to use this Coord action (it has a small penalty). Interesting method: agents can decide when to coordinate instead of exchange irrelevant messages all the time. They’ve tried with many different scenarios.

Abstraction Pathologies in Extensive Games
Kevin Waugh, Dave Schnizlein, Michael Bowling, Duane Szafron

Talking about poker competition for agents. Just two-playesrs. They use abstractions and the agent has to decice when to refine. Test with no-limit and leduc hold’em (small game, 6 cards deck, one ard ‘hidden’ and the other public). Boring… talking about the details of the game and many, many results.

State-Coupled Replicator Dynamics
Daniel Hennes, Karl Tuyls

Using evolutionary game theory, but it is single state dynamic, so it has to be extended to multi-state. Showing the behaviour in different classic games (as Prisoner’s Dilemma). Definitely…. i’m not interesting on this at all.

Wait a minute, with the examples I’ve seen hat it’s very similiar to our model of agreement, at least how it behaves. I’ll need to take a look to it. Too formal, but I hope that Alberto could help us with this.

Blogged with the Flock Browser

Tags: , ,

AAMAS09 MABS and emergent behaviour

This is my first ‘normal session’ in the AAMAS. Yesterday we had a very boring poster session: three hours standing up just to talk 10 mins. with 4 people :-( And today an stressing morning with 6 parallel sessions Impossible to attend the all the topics I’m interested in.

On the Significance of Synchroneity in Emergent Systems
Adam Campbell, Annie Wu

I’ve arrive late, so I’m only trying to get a chair.

On Recursive Simulation
Latek Maciej, Rob Axtell, Bogumil Kaminski

Using something calles n-th order rationality (I haven’t the foggiest idea about what it is). Oh!, it’s about rationality in games (I don’t like game theory and any utility-based solution). They have a tactical model (that represents the evolution of the environment, without decision making information from other agents). So only the ‘trajectories’ of the policies can be found, nor the policies themselves. A n-th order rationality is defined recursively and… ups, too fast… he’s in the example now :-( playing Blotto (I don’t know this game). He’s talking about cognitive capabilities but, at the end, it’s represented just by one equation. Can knowledge be reduce just to one equation?

Adaptive Learning in Complex Evolving Trade Networks
Tomas Klos, Bart Nooteboom

Motivation: task allocation in networks of trading agents, with input/aoutput relations. IN classical economics, individual nodes are optimized, whereas transaction costr economics is focus on transactions (edges). Agents are rational and opportunistic. Buyers choose make or buy something and it’s implemented using Gale-Shapley algorithm for matching (game theory again… maybe the title of this session is wrong), using preferences based on scores related with potential profit and some trust (loyalty) measure.

Some experimental results, that turn into something interesting when they begin to consider the network itself, at least the indegree and outdegree (4 max. with a random network I guess). So, they have a model to simulate organizations using agents to check hypothesis.

A Mathematical Analysis of Collective Cognitive Convergence
Van Parunak

The idea of CCC is intgeresting: how a ‘closed’ collective can ‘corrupt’ the knowledge being something ‘endemic’. And they’re using agents-based models to explain why this is happening. They have simulated that and now they have a formal model of all this stuff.

An interesting result in theorem 4: it detects when the system converges (non deseable), so it can be corrected. And this convergence depends on number of agents, the number of topics and the topic’s density.

Emergent Service Provisioning and Demand Estimation through Self-Organizing Agent Communities
Mariusz Jacyno, Seth Bullock, Michael Luck, Terry Payne

A simulation model to match supply with demand in service based communities (coalitions and teams). Very difficult to synchronise choices in a centralised way and it leads to over-provisioning of services (againt a reason to create a distributed SF federation and, why not,  using a small world model ;-) This work is based on the emergent behaviour of insect colonies (OMG, ants again!) -> limited knowledge about peers and local behaviour (it sounds to me).

One interesting idea: to change the type of a provided service  has a cost, a penalty, so the system trends to keep the services unchanged as long as possible. Besides, resources are limited so a limited-size registry of known providers are maintained by customers, whereas providers have a limited-seize registry of user’s requests.

The simulation considers the amount of memory and only two types of services are available. When you have too few or too much memory the performance is worse that in a medium case, where you know enough providers/customers to work locally without service changes.

Effective Tag Mechanisms for Evolving Cooperation
Matthew Matlock, Sandip Sen

The last one, about considering expertise in a agent network using tag mechanism. Tags are a useful mechanism to promote collaborative behavior and it allows to reuse knowledge (game theory again). They introduce a boolean function to tag matching and examines four models to extend the tags.

  • matched:
  • pay off:
  • paired reproduction. solves both problems, but they need more research to check if it is robust enough.

(check the paper to complete)

Blogged with the Flock Browser

Small World for Agent Search

posteraamas09 Es el título de mi póster en el AAMAS ’09. Se trata de evaluar si es válido un modelo de red de tipo small world para distribuir a un grupo de plataformas  de agentes en una red de forma que se pueda localizar fácilmente dónde se encuentra un agente con el que nos queremos comunicar.

El modelo de red que se emplea ha sido propuesto por Kleinberg y garantiza que se pouede realizar un proceso de búsqueda voraz (tomando decisiones de forma local y sin volver atrás) acotado. A este tipo de redes se les llama redes navegables.

En un artículo más extenso lo he comparado con otros modelos de redes de tipo small world y la verdad es que sale bastante bien parado: es una red tan buena como la mejor (el modelo de Barabasi según mis pruebas) en cuanto a tolerancia a fallos (destrucción de enlaces en la red), pero mucho mejor en cuanto a la búsqueda, mejorando incluso a los modelos P2P.


REBOLLO, M.: Small World Model for Agent Search (Short Paper).- In Proc. of 8th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS 2009), Decker, Sichman, Sierra and Castelfranchi (eds.), May, 10–15, 2009, Budapest, Hungarytions.