DTS311TC FINAL YEAR PROJECT

The application of a trust model based on confidence and reputation to NPC trust assessment and decision- making in market simulation games

Proposal Report

In Partial Fulfillment of the Requirements for the Degree of Bachelor of Data science and big data technology

Abstract

Trust is crucial in multi-agent systems and is a key factor affecting decision-making. In market trading scenarios in particular, agents must assess each other's reliability in order to  conduct  subsequent  transactions.  This  project  aims  to  develop  a  framework  with dynamic trust calculation at its core and apply it to the trust assessment of NPCs in a market  simulation  game.  A  dynamically  adjusted  trust  value  calculation  method  is constructed by combining trust (based on direct interaction data) and reputation (based on feedback information shared between NPCs). Compared with the traditional static model, the dynamic trust mechanism in this study can respond to changes in player behavior in real time, thereby more realistically simulating complex interaction scenarios. The focus of this project is on the generation of trust values, the update algorithm and the optimization of its performance. The specific decision-making behavior of the NPC is a secondary   implementation.   Experimental verification is used to demonstrate   the applicability and performance of the model in a dynamic game environment.

Contents

1  Introduction

1.1  Introduction, Background

Trust  computing,  as  an important research  direction in multi-agent  systems, provides theoretical support for modeling complex interactive behaviors. Trust plays an important role in the effective interaction of multi-agent systems.  However, most existing systems rely on static or rule-based decision models, which cannot adapt to complex real-time player behaviors, resulting in an interaction experience that lacks realism. In recent years, models  capable  of  dynamically   computing  trust  have   shown  strong  potential  for application in multi-agent systems.  [1]These models can adapt to different interaction environments.

In this project, a dynamic trust calculation model is applied to a market simulation game, enabling NPC to dynamically assess the player's trustworthiness based on the player's behavioral data and environmental context, thereby providing more effective support for the trading process. This model enables the NPC in the game to continuously adjust their decisions during interactions, thereby improving the player experience.

1.2  Scope and Objectives

1.2.1      Scope

- Develop a computational trust model for a market-oriented simulation game.

- Design a lightweight dynamic trust value calculation framework.

- Focus the main research on trust value calculation, and implement NPC decision logic as a secondary task.

1.2.2      Objectives

-Design  a  confidence  and  reputation  calculation  algorithm  based  on  fuzzy  logic  and reputation aggregation.

-Develop a dynamic trust value update mechanism that responds to player behavior. -Verify the accuracy and applicability of the model in a simulated environment

2  Literature Review

2.1     Related work

What is a trust model? The method to specify, evaluate and set up trust relationships amongst entities for calculating trust is referred as the trust model. Trust modeling is the technical approach used to represent trust for the purpose of digital processing.[2]

Marsh (1994) was one of the first scholars to formalize trust in computing systems. In his approach, he  integrated  various  aspects  of trust  from  disciplines  such  as  economics, psychology,  philosophy,   and  sociology.  Since  then,  many  trust  models  have  been constructed  for  various  computing  paradigms  such  as  ubiquitous  computing,  P2P networks, and multi-agent systems. [2][3]

2.2   Trust models in multi-agent systems

Trust  computing  plays  an  important  role  in  multi-agent  systems,  providing  core theoretical  support for the modeling of complex interactions and intelligent  decision- making.[4]

The  earliest trust  computing models were usually  static,  for  example,  assessing  trust values through  fixed rules  or  a  single  calculation  based  on past  data.  Dynamic  trust models   overcome  the   shortcomings   of  static   models   through   real-time  updating mechanisms.[5]

In order to cope with the needs of multi-agent interactions in complex environments, multi-dimensional trust assessment models have gradually emerged. [6] For example, in addition to the traditional confidence and reputation dimensions, researchers have also introduced  factors  such  as  social  relationships   and  risk   assessment.  These  models improve  the  accuracy  and  applicability  of trust  values  by  weighting  and  fusing  the evaluation results of multiple dimensions. [7]

Currently, the interest in trust models is not decreasing. We can see that the number of various models present in the literature is increasing. [8]. People can apply these models specifically to different scenarios to help the development of projects. I think my research on trust models is relevant.

2.3  Trust model selection

Many models for calculating trust have been developed. For example, Marsh (1994) first proposed a formal calculation framework for trust, which laid the theoretical foundation for quantifying trust values. He believes that trust is a value between -1 and 1, and his calculation method considers the risk of interaction and the ability of the interaction partner to calculate. [9] However, these concepts are not given any precise grounding, and past experience and reputation values are not considered.

Reputation symbolizes trust, and the level of ability is collected from the social network in which the  agent  is  located.  The  main value  of this model  is to use  reputation to symbolize trust, but this assessment is too simple. [10] There is also a probability method used to build the model, which takes into account past experience and reputation, but it does not significantly help in understanding the decision-making of the agent. [11]

We have chosen a trust model based on reputation and confidence. For the first time, confidence and reputation are combined for trust modeling, providing a context-aware trust  calculation  method.  A   specific  algorithmic  framework  is  provided  to  support dynamic weight adjustment. [12]It can be used well in market simulation games.

2.4   Trust computing in games

The application of the trust computing model in games is mainly reflected in two aspects: improving the intelligent behavior of NPCs and enhancing the interactive experience of players,  especially  in  real-time  interactions  in  dynamic   environments  and  complex decision-making  scenarios. For  example, market  simulation  games,  as  a  typical  open interactive scenario, provide a broad  space for the application of the trust computing model.

The evaluation of player trust in games is also very important. The application of the trust computing  model  in  games  is  conducive  to  achieving  fairness  in  online  games  and reducing the spread of untrustworthy information among players. [13]

3  Project Plan

3.1  Proposed Solution / Methodology

3.1.1      data preparation

1. Direct Interaction Data (Confidence):

- Get the player's actual performance on a specific issue from the historical case base CB.

- Key information for each interaction includes:

- Issue assignmentsO = {x1      = v1     , x2      = v2     , …}；

- Execution results O ′ = {x1      = v1 ′   , x2      = v2 ′   , …}；

- Timestamp  t

- Use a utility functionux(v)to evaluate the utility of each issue value.

2. Indirect Interaction Data (Reputation):

Reputation information collected from the social network of agents.

3.1.2       Confidence Calculation

Confidence measures the reliability of the target agent based on direct interactions. The process includes:

(1) Obtain the distribution of utility changes from historical data

Extract  the  distribution  of utility  changes  for  a  issue  x  from  the  interaction  records ΔUx    =Ux    (v′)−Ux    (v)  ,  where  v is the value  agreed in the contract and v' is the actual implementation result.

(2) Estimate Confidence Interval

Determine  the  confidence  interval  [v− ,v+]    for  the  agent’s  possible  performance  on issue x based on historical data.

(3) Fuzzify Assessment

Map the confidence interval to linguistic labels L={Poor,Average,Good} and assign confidence levels C(x,L) to each label.

(4) Compute Expected Value Range

Using confidence levels, calculate the expected value range for issue x:

where  is the membership function for label L.

（5）Calculate Maximum Utility Loss

Within the expected value range, calculate the maximum utility loss:

(6) Derive Confidence Trust Value

Based on the maximum utility loss, compute the trust value for issue based on confidence:

3.1.3      Reputation calculation

The reputation value measures indirect information collected from other agents and is calculated as follows:

(1) Obtain the reputation value distribution Rep(x, L) of the target agent on issue x from the social network, where L is the fuzzy set label.

(2) Calculate expected value range

Similar to confidence level, based on the reputation value distribution, calculate the expected value range for issue x.

(3) Calculate maximum utility loss

Calculate maximum utility loss based on reputation value:

(4) Calculate reputation trust value

Calculate the reputation trust value of the target agent on issue x based on the maximum utility loss.

3.1.4         Combining confidence and reputation

In practical scenarios, confidence and reputation are often used in combination.  The combination process is as follows:

（1）Determine the weight

∣CB ∣ ：Number of interactions in history.

θmin     ：: Confidence threshold, the minimum number of interactions that indicate that the confidence value completely dominates trust.

(2) Calculate the comprehensive expected value range

Combine confidence level and reputation to calculate the comprehensive expected value range for issue x.

Further information on the expected value range:

(3) Calculate the overall trust value

Calculate the maximum utility loss based on the overall expected value range:

The final overall trust score is obtained:

3.1.5        Construction of the trust model

(1)Confidence is the only source of trust (Trust = Confidence)

In this case, only direct interactions are considered to be a valid source of information for measuring the performance of another agent. The first contract will be full of uncertainty, and this definition of trust will only work  effectively when there have been  enough interactions.

Then the trust value for issue x is defined as:

(2)Reputation as the only source of trust (Trust = Reputation)

When the number of interactions is small, confidence cannot provide sufficient information, and reputation information may be more useful. This is a common situation

The trust value of issue x is defined as:

(3)Combining confidence and reputation (Trust = Confidence and Reputation)

In most cases, it is more reasonable to combine confidence and reputation. The logic is that as interactions between agents increase, npc will become more and more dependent on their own confidence measurement, rather than the reputation information provided by others (because direct interactions are usually more accurate than indirect information).

Finally, the trust value of issue x is defined as:

Our definition of trust (especially the last approach) views trust as a dynamic and rational concept.

3.1.6       Decision-making framework:

The comprehensive trust valueT(β, X） is compared with the preset threshold.

If T(β, X） is greater than the preset threshold, the NPC accepts the player's transaction request.

If T(β, X）is less than the preset threshold, the NPC rejects the transaction.

3.2  Experimental Design

Flowchart

Figure 1 Decision-making process flow chart

3.2.1     Testbed architecture

Design a test platform. to simulate the interaction between NPCs and players, and test the NPC decision-making mechanism based on the trust model (confidence, reputation, and comprehensive trustworthiness) to ensure the rationality and dynamic adaptability of the decision-making.

Figure2 Testbed architecture （1）

Figure3 Testbed architecture （2）

The testbed architecture has four components: the simulation engine, the database, the user interface, and the agent framework.

The simulation engine is responsible for starting the game, controlling the simulation environment by adjusting parameters, and managing processes such as player requests, NPC trust calculations, and decision execution.

The database stores environment and agent data. This testbed also provides the ability to record other data types in the database, as well as data replay and analysis tools.

The user interface provides real-time visualization of NPC-player interactions, trust value changes, and decision results. Figure 4 shows a game monitoring interface.

The agent skeleton is designed to allow the implantation of custom internal trust representations and trust revision algorithms.  The  Java  classes  that  define  the  agent skeleton implement all the necessary interfaces to allow agent-agent interactions (via the simulation  engine).  The  agent  skeleton  also  handles  coordination  tasks  with  the simulation engine, such as opinion formation and evaluation calculations. In the future, it may be possible to try to develop agent skeletons in other programming languages to provide more flexibility for agent designers.

Figure4 Game monitoring interface.

3.2.2       Overview of the process

1.  The player  sends  a  request  to  the  NPC  (e.g.,  a  quest,  a  trade,  etc.).  The  player's behavior. can be designed to be honest, dishonest, or mixed.

2. The simulation engine assigns the request, and the NPC calculates the player's overall trustworthiness.

3. The NPC makes a decision to accept or reject based on the trustworthiness and the set threshold.

4. The player performs the task, and the simulation engine records the task result and updates the trust value.

5.  The  data  storage  module  records  the  interaction  data  and  provides  analysis  and playback functions.

3.2.3       Test indicators

1. Change in confidence trend: Whether the dynamic changes in confidence, reputation and overall confidence in different player behavior. patterns are as expected.

2. Decision  accuracy:  Consistency between the NPC's  acceptance/rejection  decisions and the player's actual behavior.

3.  Model  adaptability:  Whether  the  model  can  dynamically  adjust  trust  values  and decisions based on player behavior. patterns

3.3  Expected Results

3.3.1      Expected results

NPC decision-making is highly consistent with player behavior. patterns:

1.   Honest   players:   gradually   increase   overall   trustworthiness   and   increase   NPC acceptance rate.

2.  Dishonest  players:   gradually  decrease   overall  trustworthiness   and  increase  NPC rejection rate.

3.   Mixed    players:   overall    trustworthiness   and    decision-making    show   dynamic fluctuations.

3.3.2       Data visualization:

Plot the trust value change curve and decision-making distribution map to demonstrate the dynamic adjustment capability of the model.

3.4  Progress Analysis and Gantt Chart

In the current school year project (FYP), the literature research on the application of multi-agent system trust models in simulated market games will be completed in October 2024, and a plan proposal will be submitted in November. From November to December, relevant knowledge and skills will be learned, relevant data will be collected, and trust models will be established and analyzed. From December 2024 to April 2025, specific experiments will be completed, the results will be evaluated and compared, and a first draft of the paper will be completed in the process. Finally, the paper will be revised and the defense will be completed.

Figure5 Gantt chart

4  Conclusion

This project  focuses  on  the  dynamic  trust  calculation  of NPCs  in  market  simulation games. By combining confidence and reputation, an efficient trust assessment framework is proposed. At the same time, its applicability and performance in dynamic scenarios are verified through experiments, providing a reference for further research in the field of game AI.

However, this study is limited to accepting or rejecting NPCs decisions. The trust model mentioned in this paper can also guide NPCs to make more complex decisions, such as   modifying the content of a transaction.If I have the opportunity, I can study it further in  the future.