Mean-Field Analysis for he Evaluaion of Gossip Proocols Rena Bakhshi, Lucia Cloh, Wan Fokkink, Boudewijn Haverkor Deparmen of Compuer Science, Vrije Universiei Amserdam, Amserdam, Neherlands Cenre for Telemaics & Informaion Technology, Universiy of Twene, Enschede, Neherlands Embedded Sysems Insiue, Eindhoven, Neherlands Absrac Gossip proocols are designed o operae in very large, decenralised neworks. A node in such a nework bases is decision o inerac (gossip) wih anoher node on is parial view of he global sysem. Because of he size of hese neworks, analysis of gossip proocols is mosly done using simulaions, ha end o be expensive in compuaion ime and memory consumpion. We employ mean-field approximaion for an analyical evaluaion of gossip proocols. Nodes in he nework are represened by small idenical sochasic models. Joining all nodes would resul in an enormous sochasic process. If he number of nodes goes o infiniy, however, mean-field analysis allows us o replace his inracably large sochasic process by a small deerminisic process. This process approximaes he behaviour of very large gossip neworks, and can be evaluaed using simple marix-vecor muliplicaions. I. INTRODUCTION We consider large-scale neworks where a large number of nodes ineracs. In such neworks, gossip proocols have shown o be a sensible paradigm for developing scalable and reliable communicaion mechanisms. For insance, informaion can be spread in a large-scale nework if nodes periodically conac each oher in a random fashion, and exchange heir local informaion. When a large number of nodes ineracs in a conneced environmen, various phenomena emerge ha canno be explained in erms of he behaviour of a single node. Therefore, we are ineresed in going from a deailed local model a node level o an absrac global model of he sysem. To sudy he emergen behaviour of gossip proocols demands he consideraion of large-scale neworks []. Thus, he analysis of gossip proocols wih auomaed ools is hard i is, for example, beyond he capabiliies of curren probabilisic model-checking ools [2]. In his paper, we show ha meanfield analysis is well suied for a formal evaluaion of gossip proocols. The sochasic process represening he modelled sysem converges o a deerminisic process if he number of nodes goes o infiniy, providing an approximaion for large numbers of nodes. A preliminary version of his paper appeared in [3]. Noably, Sec. IV of he curren paper presens an analysis of basic GTP, whereas [3] resriced he analysis o only he hop-coun meric wihin basic GTP. This paper is furher organised as follows. Sec. II gives a brief overview of he gossip paradigm, and explains an insance of such a proocol, ha is, a basic version of he gossiping ime proocol (GTP). In Sec. III, we describe he necessary mean-field heory, and devise a simple analyical model for gossip-based informaion disseminaion as an illusraive example. In Sec. IV we presen an analysis of basic GTP using he mean-field convergence resul from Sec. III. Sec. V discusses relaed work. Sec. VI concludes our paper. II. GOSSIP PROTOCOLS Gossip-based proocols (someimes referred o as epidemic proocols) are appealing in large-scale decenralised sysems. In hese proocols, nodes exchange daa in a random fashion: a node chooses wih some probabiliy a peer o exchange informaion wih. The gossip concep has originally been proposed for he analysis of daabase replicaion schemes [4]. A. A Generic Gossip Proocol Figure illusraes he skeleon of a generic gossip-based proocol. Each node has a local sae s and execues wo differen hreads, an acive and a passive one. The acive hread periodically iniiaes a sae exchange wih a random peer p by sending i a message conaining he local sae s, afer which i wais for a response. The passive hread wais for a message sen by an iniiaor and replies o i wih is local sae. The random peer selecion is based on he se of neighbours as deermined by a membership proocol (e.g., []). while rue do wai ( ime unis) p RandomPeer(); prepare(s); send s o p; s p receive( ); s Updae(s, s p ); (a) acive hread Fig.. while rue do s p receive( ); prepare(s); send s o sender(s p ); s Updae(s, s p ); (b) passive hread The skeleon of a gossip proocol For a pair of nodes A and B, where A is he acive node and B is he passive one, we describe he proocol from he poin of view of each paricipaing node. In paricular, node A picks a neighbour B a random (mehod RandomPeer()) afer a no necessarily consan ime span of lengh, and iniiaes he sae exchange (gossip) wih i. I does so by sending (a par of) is local sae s o B, and wais for B s response.
Upon receip of he response, node A updaes is local sae (according o he mehod Updae(s, s p )). In response o being conaced by A, node B sends (par of) is local sae o A and updaes is local sae accordingly (mehod Updae(s, s p )). Mehod Updae is proocol specific. I updaes he local sae of a node based on he previous local sae, and he sae informaion received from he random gossip parner. In gossip-based informaion disseminaion proocols (as in, e.g., disribued news service proocols [5], [6]), a finie lis of daa iems (e.g., news iems), called he cache, composes he local sae of a node. The generic operaion prepare(s) in Figure is replaced by an operaion s RandomIems(). The mehod Updae merges he lis of old iems wih he lis of received iems. The measures of ineres of hese proocols include he number of copies of a daa in he nework afer some ime and he amoun of ime needed for he daa o spread in he nework. In gossip-based membership managemen proocols, a finie se of peer addresses, called he parial view, comprises he local sae of a node. The mehod Updae (as in [7], [8]) creaes a new sae hrough a sample of he union of he old and he received views. The performance merics of hese proocols include a disribuion of he parial view size, he number of nodes reached in he presence of nodes failures. In probabilisic broadcasing (e.g., [9]), he sae of a node is a flag ha records wheher he node is infeced. Mehod Updae ses he sae o infeced if he received sae is infeced. The performance of hese proocols can be measured in, e.g., he ime unil all nodes receive he broadcas message. In gossipbased disribued aggregaion (e.g. []), he sae of a node is a numeric value, which can be any parameer of he environmen, such as a emperaure or he curren load. All values a nodes conribue o an aggregae value, compued using some aggregaion funcion, for insance, average, sum, ec. The mehod Updae simply reurns he resul of he aggregaion funcion. For hese proocols, a general measure of ineres is he convergence of resuls of he aggregaion funcion, bu oher measures depend on he aggregaion funcion chosen. We refer o [] for a horough survey on gossip applicaions. The sae exchange beween nodes can be implemened as one of he following policies: only he node ha iniiaes a gossip sends sae informaion o is parner (push), a nodeiniiaor requess sae daa from is gossip parer (pull), boh nodes send heir sae informaion o each oher (push-pull). B. Gossiping Time Proocol Proocols based on epidemic and gossip conceps have found various pracical applicaions [], including non radiional gossip applicaions [2], such as gossip-based clock synchronisaion. The Gossiping Time Proocol (GTP) [3], [4] is a self-managing gossip ime synchronisaion proocol for peer-o-peer neworks. The proocol operaes in a nework of nodes, each of which equipped wih a local clock, and assumes he presence of a leas one node wih accurae and robus ime in he nework. Time is disseminaed hroughou he nework by leing nodes periodically gossip heir clock samples. Tha is, each node periodically selecs (iniiaes a gossip wih) a random peer from he nework o exchange ime informaion wih. The iniiaing period is deermined by a value of he gossip delay parameer, which is he curren delay beween subsequen gossip ineracions. The nodes subsequenly exchange heir local seings such ha aferwards he node wih he worsequaliy ime has adoped he higher-qualiy ime of he oher node. The proocol assumes a presence of he peer-sampling service [], which allows a node o conac a uniformly randomly seleced alive node. In basic GTP, he qualiy of he ime sample a a node is based on he disance from he ime source o he node (hop coun meric), ha is, he number of nodes on he synchronisaion pah from he node o he ime source. The ime source has hop coun equal o. Compleely unsynchronised nodes have a hop coun. A gossiping node rejecs he ime sample if he hop coun of is gossip parner is no smaller han is own. Furhermore, a node adops a ime sample if i has no been synchronised for a long ime. Tha is, if he difference beween he las updae and he curren ime is larger han a imeou period, hen a node acceps a ime sample even hough i may degrade is ime qualiy (wih respec o he hop coun meric). Concisely, if he node decides o accep he sample, i synchronises is clock, and updaes he values of he local variables. Namely, he node records he value of curren ime as he ime of he las clock updae, and ses he hop coun o he value of he gossip parner hop coun incremened by one. The GTP proocol parameers described above are sored as he following local variables, according o [4]: a gossip delay as GOSSIPING_DELAY, a ime of he las clock updae as LAST_UPDATE, a hop coun as TS_DISTANCE, a imeou as _STANDALONE_PERIOD_. Furhermore, each node may decide o adap a rae a which i iniiaes a imesamp exchange (gossip frequency) based on is local seings. For insance, he beer synchronised he node is, he lower he gossip frequency i may assume. In doing so, he gossip frequency gradually decreases when he nework is synchronised and sable. Noe ha dynamic gossip frequency is beyond basic GTP. Our goal is o show how a mean-field framework can be applied o gossip proocols, on he example of he basic GTP proocol. Tha is, nodes execue basic GTP based on an immediae clock adjusmen model, and change gossip frequencies, depending on a gossip delay. For he original proocol and is design deails, we refer o [3], [4]. III. MEAN-FIELD MODELLING AND CONVERGENCE This secion inroduces he heory needed o apply mean field resuls o gossip proocols. We say close o he presenaion in [5] bu change noaions when appropriae and simplify hings if possible in he gossip conex. A. Modelling and Convergence Resul A discree-ime Markov chain (DTMC) is a sochasic process {Y () N} ha akes values in a counable sae
p N (m ) p N (m ) Fig. 2. A single node space S. A DTMC obeys he Markov propery, ha is, he nex sae is independen of he pas, given he presen sae: Pr{Y ( + ) = j Y () = i,..., Y () = i } = Pr{Y ( + ) = j Y () = i }, i l, j S. We consider a sysem of N N ineracing objecs ha are idenically defined. The objec wih index n {,..., N} is represened by he discree-ime sochasic process {X N n () N} which akes values in he se S = {,..., K } where K = S is he number of differen saes. Example In a gossip nework, a node is represened by an ineracing objec. As a running example we consider a simple informaion disseminaion proocol. A piece of informaion, e.g., he curren ime, is forwarded hrough he ne. A node can be in one of wo saes: eiher i already has he informaion (sae ) or i is no ye informed (sae ). Hence, he sae space for a node is S = {, } wih S = K = 2. Le m be a fracion of informed nodes, and p N (m ), he probabiliy of moving from sae o sae. Figure 2 shows a graphical represenaion of he sae-ransiion diagram describing such a node; he possible ransiions and heir probabiliies will be explained laer in his secion. The complee sysem is composed of he N objecs and is, consequenly, also described by a discree-ime sochasic process: Y N () = ( X N (),..., X N N ()). Is sae space is S N which has S N = K N elemens. For he mean-field convergence resul we assume ha we can no disinguish objecs ha are in he same sae. I hen suffices o keep rack of he fracion of objecs in each sae. These fracions are colleced in anoher sochasic process M N () = (M (),..., M K ()) called he occupancy measure. Is elemens are defined as M N i () = N N {X N n ()=i}, i S, n= where {X N n ()=i} is if Xn N () = i and oherwise. Is sae space SM N RK has ( ) S N M K + = K elemens (he number of ways o disribue N objecs over he K saes hey can be in). One sae from his sae space is denoed m = (m, m,..., m K ) S N M, where m i is he fracion of nodes in he sae i. () Example 2 For he informaion disseminaion example, he sae space of he occupancy measure is {( k SM N = N, k ) } k {,..., N}. N Is size is ( ) S N M 2 + = = N +. 2 The evoluion of he sysem of ineracing objecs is described by he local ransiion probabiliies of each objec. The nex sae of any objec does no only depend on he curren sae of he objec bu also on he curren occupancy measure m: P N i,j(m) = Pr{X N n ( + ) = j X N n () = i, M N () = m}, i, j S, m S N M. These probabiliies are he same for all objecs. They are gahered ino he ransiion probabiliy marix P N (m). These local ransiion probabiliies deermine he unique ransiion probabiliy marix for he global sysem Y N (), which is a DTMC because is nex sae (=occupancy measure) only depends on he curren sae. Example 3 A node can only move from being uninformed (sae ) o being informed (sae ). Aferwards i says in sae forever, ha is, i never forges. Suppose ha in each ime sep a node A iniiaes a gossip ineracion wih probabiliy g. I randomly chooses a parner node B among he oher nodes. If B is already informed and A is no, A moves o sae, so ha we model a simple pull proocol. Noe ha m is he fracion of informed nodes in he sysem and m = m he fracion of uninformed nodes. The oal probabiliy for moving from sae o sae equals p N (m ) = P N,((m, m )) = g m N. Here, m N is he number of informed nodes and m N/(N ) is he probabiliy ha a node chooses an informed node ou of he possible nodes (i does no pick iself) as gossip parner. The complee probabiliy marix is hen given by ( P N ((m, m )) = p N (m ) p N (m ) ). For he global sysem, he probabiliy o move from a fracion of m informed nodes o m informed nodes, for m m, equals ( ) m N (p N (m m (m ) ) (m m)n ( p N (m ) ) m N, ) N where m = m, m = m. This binomial expression is composed of he number of possibiliies o choose exacly he missing (m m ) N objecs ou of he m N uninformed nodes, hese hen all have o ake he ransiion o sae, and all oher m N nodes remain in sae. Consider now he occupancy measure M N () of he sysem a a given finie ime N. Recall ha M N () is a random
Pr{M N () m }.8.6.4 N=, DTMC simulaion N=, DTMC and.2 simulaion N=, DTMC and simulaion mean field..2.3.4.5 m Fig. 3. Disribuion of M N () for g =. and M N () = (.,.99) variable. For a given iniial occupancy measure m N, here are wo ways o deermine he disribuion of M N (): firs, we can calculae he ransien disribuion analyically a ime, requiring vecor-marix muliplicaions wih a vecor of size SM N. Second, we can employ discree-even simulaion o esimae he disribuion. Ofen only discree-even simulaion is possible since, for large N, he size of he sae space makes he analyical compuaion of he ransien probabiliies pracically infeasible. Bu even discree-even simulaion of his large DTMC is expensive. Example 4 Figure 3 shows he analyically compued (using mean-field analysis) disribuion of he fracion of informed nodes a ime =, for gossip probabiliy g =. and iniial occupancy measure M N () = (.,.99). Noe ha he disribuion is more deerminisic for larger N. We also simulaed his simple disseminaion proocol in a round-based fashion similar o simulaions in PeerSim [6]. In one round, which equals one ime sep, each uninformed node gossips wih probabiliy g and picks a random peer. If his peer is already informed, he number of informed nodes for he nex round is increased by one. Using independen runs for each curve, he resuling disribuions for M N () are also shown in Figure 3. A his poin, he so-called mean-field convergence resul applies. I capures he limiing behaviour of he complee sysem if he number of objecs N goes o infiniy and so provides an approximaion for he occupancy measure for large N. The requiremen is ha for all local saes i, j S, all m R K and for N P N i,j (m) converges uniformly in m o some P i,j (m), which is a coninuous funcion of m. If his requiremen is saisfied, he occupancy measure converges almos surely o a deerminisic limi. This means ha in case N for each local sae i he fracion Mi N () of objecs in sae i a ime is known wih probabiliy one. A sequence f N of real valued funcions converges uniformly wih limi f if for every ε > here exiss a naural number n such ha for all x and all N n we have f N (x) f(x) < ε. Theorem (cf. [5]) Fix he iniial occupancy measure o be idenical for all N N: M N () = µ(). Define he limi of he local probabiliy marix: P (m) = lim N P N (m), m R K. Define he deerminisic process Then for any N, µ( + ) = µ() P (µ()). lim M N () = µ(), wih probabiliy, N ha is, µ() is he deerminisic limi occupancy measure for N. For large N we can now approximae he sochasic process for he occupancy measure by his deerminisic process. Example 5 The limi of he probabiliy o move from sae o sae is p(m ) = lim N g m N = g m, which is coninuous in m. The requiremen for he applicaion of he mean-field convergence resul is hus saisfied. If we se µ() = (.,.99) and g =., he deerminisic limi for ime = is µ() = (.256,.9744) compued by en marix-vecor muliplicaions. I is indicaed by he verical line for m in Figure 3. B. A Mehodology for he Mean-field Analysis of Gossip Proocols We summarise how mean-field analysis can be used for he performance evaluaion of gossiping proocols. Our mehodology consiss of he following seps: Sep Formal descripion: The formal specificaion of a sysem helps o obain no only a beer (more modular) descripion, bu also a clear undersanding and an absrac view of he sysem. In general, i is hard o give a full specificaion of a sysem or proocol under sudy. Such a sudy is usually done on a simplified sysem model of he acual proocol: one has o decide which characerisics of he proocol should be sudied, and which parameers of he proocol should be modelled in order o sudy hese characerisics. In order o simplify he sysem model, assumpions should be made. These assumpions should be suppored by experimenal sudy. Sep 2 Idenificaion of local saes and ransiions: This sep requires o idenify he se S of local saes of a node. The saes should reflec all relevan siuaions a node can be in. Transiions beween local saes usually occur because of gossip ineracions.
Sep 3 Transiion probabiliies: The (local) ransiion probabiliies depend on he global sae of he gossip nework model. The probabiliies have o be invesigaed horoughly. A node migh also behave inrinsically in a probabilisic way. A he end of his sep sands a direcive of how o compue he ransiion probabiliy marix depending on he curren global sae. Sep 4 Mean-field convergence requiremens: Only if he local ransiion probabiliies converge appropriaely for N we can successfully apply he mean-field convergence heorem. Sep 5 Mean-field limi: Finally, we can compue he mean-field limi for our model using Theorem. Wih he obained resuls we can es and compare differen designs. IV. A MEAN-FIELD MODEL FOR BASIC GTP A deailed descripion of basic GTP can be found in Sec. II-B and in [3], [4]. This corresponds o Sep in our mehodology. Sep 2 and Sep 3 are accomplished in he following hree subsecions (IV-A C). Secion IV-D corresponds o Seps 4 and 5. A. Sae Space The sae of a node in a basic GTP nework is given by a riple (g, l, h). The firs componen g denoes he gossip delay. When i is equal o zero a node iniiaes a gossip ineracion. The second componen l represens a couner for he las updae. In GTP, he ime of he las clock updae is sored and if necessary compared o he curren ime. If he difference exceeds he sandalone period, an updae is enforced. We replace i wih a couner which is se o he lengh of he sandalone period a every updae. Reaching zero, a clock updae is enforced a he nex ineracion. Finally, h is he number of hops he iming informaion has ravelled from he ime source. Le G max be he maximal gossip delay, and le L be he sandalone period. We inroduce H o be he maximal hop coun recorded. A node in a sae wih h = H has a hop coun of a leas H. A node wih h = is said o be unsynchronised. The sae space of single node hen is S = {,..., G max } {,..., L} {,..., H, }, which is of size S = (G max + )(L + )(H + 2). B. Gossip Delay Though basic GTP has a fixed gossip delay, we design he model in such a way ha i allows for he gossip delay o vary, depending on he hop coun of a node. We assume ha here is a funcion G : {,..., H, } {,..., G max } ha gives he gossip delay G(h) for any hop coun h. C. Local Transiion Probabiliies The behaviour of a single node is deermined by is sae and he curren occupancy measure m. In he sequel, we use a kind of paern maching noaion: for example, (g, l, h g > ) denoes any sae where g > while l and h are chosen arbirarily from heir respecive value ses. The expression m (g,l,h h<h) is an example for he abbreviaion of a sum of occupancy fracions, defined by m (g,l,h h<h) = m (g,l,h). g l h<h Time Sources: We begin wih he descripion of he behaviour of a ime source, ha is, a node in a sae wih h =. Time sources never updae heir clock, hence, componen l has no meaning and we always se i o be L. If he gossip delay is larger han zero (g > ), we jus decremen i by one. If i is equal o zero, he gossip delay is rese o G(). P N (g,l, g>),(g,l,) (m) = P(,l,),(G(),L,) N (m) =. As one can see, ime sources ac independenly of heir environmen. Acive Nodes: If he gossip delay g of a node A is equal o zero, i becomes acive and iniiaes a gossip ineracion wih a peer B randomly chosen from he remaining nodes. In his ineracion, he clock of A migh ge updaed. In GTP, an ineracion is discarded if during is course here has been anoher ineracion leading o an updae of he clock. In he model we require ha for each node only one ineracion can be acive, oherwise we say ha here is a collision. An updae can only ake place if he ineracion prevails, ha is, no collision occurs. Afer A has chosen a suiable peer B, he probabiliy noc N (m) ha here is no collision is given by he probabiliy ha all oher acive nodes selec peers differen from A and B. The probabiliy ha a node chooses neiher A nor B (given ha i does no ry o inerac wih iself) is (N 3)/(). We consequenly have noc N (m) = ( N 3 ) m(,l,h) N. We furher have o disinguish beween nodes wih an enforced updae (l = ) and hose wihou. If an updae is enforced, he clock will be updaed as long as he peer is synchronised, having a hop coun h differen from. If he updae is opional, he clock value is only changed if his does no increase he hop coun, ha is, if h < h. In eiher case, he new sae afer a successful updae is (G(h + ), L, h + ). The probabiliy o selec a passive peer wih hop coun h is m (g,l,h g>) N/() and so he probabiliy of a successful updae is P(,,h h>),(g(h N ),L,h +) (m) = m (g,l,h g >) N noc N (m), h <, P(,l,h l>,h>),(g(h N ),L,h +) (m) = m (g,l,h g >) N noc N (m), h < h. Noe ha hese probabiliies are no correc if he new hop coun is H. Since H subsumes all hop couns of a leas H, he probabiliy would be slighly differen. We omi his special
case here and in he following presenaion, hough i has of course been included in he model when compuing he resuls. I remains he case where he ineracion does no lead o an updae of he clock. This can happen if () a collision occurs when gossiping wih a passive node, (2) an acive node is seleced as peer, also leading o a collision, or (3) he ineracion has prevailed bu he peer canno provide a suiable hop coun. We again disinguish enforced and opional updaes and ge he following probabiliies. P(,,h h>),(g(h),,h) N (m) = m (g,l,h g >) N ( noc N (m)) + m (,l,h ) N + m (g,l, g >) N noc N (m), P(,l,h l>,h>),(g(h),l,h) N (m) = m (g,l,h g >) N ( noc N (m)) + m (,l,h ) N + m (g,l,h g >,h h) N noc N (m). Passive Nodes: A passive node wih g > has o be conaced by an acive peer wih hop coun h o be able o updae is hop coun o h +. This happens wih probabiliy m (,l,h ) N/(N ). The gossip delay is decreased by one in all cases, shorening he ime unil he nex gossip iniiaion. Following he same line of argumenaion as for acive nodes, he probabiliies for successful ineracions are P(g,,h g>,h>),(g(h N +),L,h +) (m) = m (,l,h ) N noc N, (m), h < P(g,l,h g>,l>,h>),(g(h N +),L,h +) (m) = m (,l,h ) N noc N (m), h < h. The probabiliy of no updaing he clock is again composed of hree erms: he probabiliy of having a collision, he probabiliy of no being chosen as a peer a all, and he probabiliy of having an ineracion wih a peer no providing a suiable hop coun, as follows: P(g,,h g>,h>),(g,,h) N (m) = m (,l,h ) N ( noc N (m)) ( + m ) (,l,h ) N + m (,l, ) N noc N (m), P(g,l,h g>,l>,h>),(g,l,h) N (m) = m (,l,h ) N ( noc N (m)) ( + m ) (,l,h ) N + m (,l,h h h) N noc N (m). D. Mean-Field Limis The probabiliy noc N (m) of having no collision converges for N : noc(m) = lim N nocn (m) ( N 3 = lim N ) m(,l,h) N = e 2 m (,l,h). For all he local ransiion probabiliies he number of nodes N only appears in he facor N/() which has limi for N, and in he expression noc N (m). The limiing probabiliies are hus easily obained by removing he facor N/() from he expressions and by replacing noc N (m) by he above limi noc(m). E. Comparison wih emulaion resuls In [4], emulaion is used o explore how GTP behaves in pracice. For basic GTP a nework of 5 nodes is emulaed on a single worksaion, using he local objec passing implemenaion for communicaion. One node is a ime source, having hop coun zero, all oher nodes are no synchronised. The gossip delay is fixed and independen of he sae of a node and se o 25 seconds. The maximum sandalone period is also se o 25 seconds. Fiing our model o his scenario, we se he fracion of nodes being a ime source o /5. We assume ha one sep in he model corresponds o one second in he emulaion. This slighly overesimaes he duraion of a gossip ineracion which is repored o be in he sub-second range. The maximum gossip delay is G max = 25 seconds and since i is fixed we have G(h) = G max for any hop coun h. The maximum delay beween wo updaes is L = 25 seconds. The maximum hop coun is chosen o be H = 5. A single node hus can assume 26 26 7 = 492 saes. The ime source fracion sars off wih g = 2, ha is, i iniiaes a gossip ineracion for he firs ime afer 2 seconds. The unsynchronised nodes have remaining gossip delays uniformly disribued beween and G max. Figure 4(a) shows he evoluion of he number of nodes ha are aware of he ime source over ime. A node becomes aware of he ime source exisence when is hop coun changes o a finie value. For he mean-field model we have muliplied he fracion of nodes wih a hop coun smaller han by 5 o obain he depiced curve. The curves of emulaion and analyical model proceed close o each oher, boh approaching 5 afer abou 2 seconds, ha is, afer abou 8 gossip cycles.
number of nodes 4 2 8 6 4 2 mean-field emulaion 2 4 6 8 2 average hop coun 2 8 6 4 2 2 4 6 8 2 number of nodes 35 3 25 2 5 5 mean field emulaion 5 5 2 hop coun (a) Discovering ime source exisence (b) Average hop coun for differen (c) Disribuion of hop coun number of ime sources Fig. 4. Comparison wih emulaion resuls (includes daa from Figures 6.(a), 6.5(b) and 6.6 in [4]) In Figure 4(b) we compare he evoluion of he average hop coun when he number of ime sources is muliplied by and, respecively. For he mean-field model, he average hop coun is compued for synchronised nodes only, hus being a kind of underesimaion as long as no all nodes are aware of he ime source exisence. For he emulaion we do no know which formula was used o compue he average hop coun while here are unsynchronised nodes presen. A he beginning, he average hop coun increases faser in he emulaion, however, mean-field model and emulaion sele o similar values. The change in he average hop coun depends logarihmically on he number of ime sources. Figure 4(c) finally shows he hisogram of hop couns afer he proocol sabilises. For he mean-field curve we have aken he disribuion a ime = 6, neglecing he fac ha here are minor oscillaions because of ime source gossip. Taking his ino accoun, and he fac ha we are no fully aware of how he disribuion was calculaed for he emulaion, here is a close mach beween he emulaion and he mean-field resul. Figure 4 documens ha he presened mean-field model capures he main feaures of basic GTP. The evoluion of he hop couns in he considered scenario is quie precisely represened. In he following we concenrae on he mean-field model. We show furher measures ha were no considered in he emulaion experimens in [4] and also evaluae he influence of varying he gossip delay. F. More properies of basic GTP We sick o he scenario of he previous secion. Following Figure 4(c) we depic in Figure 5(a) he evoluion of he hop coun disribuion over he firs minues. Each curve corresponds o he fracion of nodes ha have a hop coun of a mos a given value. The disance beween wo curves corresponds o he fracion of nodes ha has exacly a given hop coun as shown exemplarily for a hop coun of seven. A he beginning, almos all nodes are unsynchronised which resuls in he area o he lef of he graph. Gradually, he nodes acquire hop couns smaller han. Since his change originaes from he ime source, hop couns differen from are relaively small a firs. Over ime, he hop coun disribuion seles o a quasi sable sae, wih on average higher hop couns han a he momen he nework go fully synchronised. Figure 5(a) suggess ha he hop coun disribuion reaches a sable sae. This is no compleely rue, since here is a periodic disurbance whenever he ime source fracion gossips every 25 seconds. Figure 5(b) shows he fracion of nodes having hop coun zero, one, or wo for he ime inerval from 3 o 5 seconds. The fracion of ime sources (hop coun ) is consan since no node wih a hop coun larger han zero can ever become a ime source. The fracion of nodes having hop coun one oscillaes: wih each ime source gossip, he fracion increases abruply, decreasing gradually aferwards. Also for hop coun wo, here is sill a visible periodiciy. The change is already very small, for higher hop couns he periodiciy effec wears off compleely. fracion of nodes.9.8.7.6.5.4.3.2. hop coun = 7 2 3 4 5 6 fracion of nodes.4.35.3.25.2.5..5 hop coun = hop coun = hop coun = 2 3 35 4 45 5 fracion of nodes.8.6.4.2 6 7 8 9 2 (a) Disribuion of hop coun (b) Periodiciy of hop coun probabiliies (c) Hop coun afer ime source fails Fig. 5. Hop couns for consan gossip delay (25 seconds)
fracion aware of imesource.9.8.7.6.5.4.3.2. 3 seconds 2 seconds 4 seconds second 5 seconds 6 seconds 2 3 4 5 6 7 per node.... second 2 seconds 6 seconds e-5 5 5 2 average hop coun 6 5 4 3 2 second 2 seconds 6 seconds 5 5 2 (a) Synchronisaion speed (b) Upgrading ineracions (c) Average hop coun Fig. 6. Varying he saic gossip delay Figure 8(a) depics he ineracion aciviy per node over he firs minues. The naure of he mean-field model makes i necessary o sae ineracions per node. Mapping back o he original scenario, muliplying he indicaed values wih 5 resuls in he oal number per second. Ineracions are gossip aemps by nodes which have reached hop coun zero. The number says consan due o he fac ha we have a consan gossip delay of 25 seconds. Only when he ime source fracion gossips, here is a small spike in he curve. A collision occurs if here is more han one aemp of a gossip ineracion wih a node. The number of collisions is also consan, being a funcion of he number of ineracions. A change means ha a node adjuss clock and hop coun o a differen value. The hop coun migh be larger han before if he updae has been been forced by he las-updae flag. A he beginning, changes are rare, since mos ineracions ake place beween unsynchronised nodes, no leading o any updaes. In he synchronisaion phase, he number of changes increases and finally seles down o a sable level. Because of enforced updaes here are always changes o be expeced. Wih upgrades we denoe changes ha acually lead o a beer hop coun. Their number is of course smaller han he number of changes, seling down o a posiive number as well: as long as here are downgrading changes because of enforced updaes, here will also be upgrading ineracions in he sequel. Finally we wan o show he behaviour of he nework when he ime source fails. We do his by iniialising he nework wih he sae afer minues (ime = 6), running wih a single ime source and hen redisribuing he fracion of nodes being a ime source o all oher saes. Figure 5(c) shows wha happens o he hop coun disribuion in he following minues. As could be expeced, nodes wih a low hop coun die ou over ime, leaving all nodes a he chosen maximum hop coun of H = 5. G. Differen saic gossip delays Wih he nex graphs we wan o clarify he influence of he gossip delay on he performance of he complee nework. In general, one expecs ha a higher gossip delay leads o a lower synchronisaion speed. On he oher hand, i also implies less communicaion. The quesion we asked ourselves was: can a small gossip delay lead o a slower synchronisaion han a higher gossip delay because of oo many ineracions and, subsequenly, collisions? Figure 6(a) shows he speed of synchronisaion for gossip delays beween and 6 seconds. In general, synchronisaion slows down wih increasing gossip delay. Bu if nodes gossip every oher second, ha is, if he gossip delay is one, he synchronisaion proceeds slower han for a delay of 2, 3, 4 or 5 seconds. In his case, collisions impede a fas disseminaion of he iming informaion hrough he nework. Figures 6(b) and 6(c) furher subsaniae his insigh. In 6(b), he upper se of curves depics he oal number of iniiaed ineracions, he lower se shows he number of ineracions really leading o an upgrade. Even hough wih a gossip delay of one second he oal number of ineracions is highes, he number of upgrades is lower han for a gossip delay of wo seconds. Figure 6(c) documens ha also he average hop coun for a gossip delay of one is higher han for a gossip delay of wo. H. Dynamic adapaion of gossip delay Our model allows o dynamically adap he gossip delay o he sae of he sysem, ha is, o is curren hop coun, via he funcion G(h). Gradual GTP also offers his possibiliy, hereby aking more parameers (no only he hop coun) ino accoun. In line wih he descripion in [4] we wan a node o gossip more ofen if is ime has bad qualiy. For our model ha ranslaes o a high hop coun. For his purpose we inroduce a minimal gossip delay G min. The gossip delay of a node is hen se o { Gmin, h =, G(h) = G max h H+ (G max G min ), h H. Unsynchronised nodes iniiae a gossip as ofen as possible while a ime source does so as seldom as possible. For all oher hop couns, he gossip delay spreads linearly beween G min and G max. Noe ha numerically a hop coun of is reaed like H +. We compare hree cases: he scenario considered so far wih G min = G max = 25 seconds, a small range of gossip delays wih G min = 5 seconds and G max = 35 seconds, and a large range wih G min = 5 seconds and G max = 45 seconds. Figure 7 depics boh he synchronisaion speed (lef se of curves) and he average hop coun (righ se of curves) in he
.8.6.4 ineracions collisions changes upgrades.8.6.4 ineracions collisions changes upgrades.8.6.4 ineracions collisions changes upgrades.2.2.2 per node..8 per node..8 per node..8.6.6.6.4.4.4.2.2.2 2 3 4 5 6 2 3 4 5 6 2 3 4 5 6 (a) G min = G max = 25 (b) G min = 5, G max = 35 (c) G min = 5, G max = 45 Fig. 8. Ineracion aciviy. firs minues. Since nodes wih a high hop coun have more chances o upgrade if he range of he gossip delay is enlarged, he synchronisaion speed is increased as opposed o he saic gossip version. Bu his higher gossiping frequency of nodes wih a high hop coun also leads o a slighly increased average hop coun. Figure 8 compares he ineracion aciviy of he hree scenarios. While for a saic gossip delay he number of ineracions is independen of he sae of he sysem, i highly depends on he sae if he gossip delay is compued dynamically. In he beginning, when mos of he nodes are unsynchronised, here are many ineracions, leading o faser synchronisaion. In he long run, he aciviy paern seles o similar values for all hree scenarios, wih a slighly increasing number of ineracions wih increasing range of gossip delay. Wih he conduced experimens we have shown how o successfully derive a large variey of useful measures for basic GTP using a mean-field model. While emulaion like in [4] requires he availabiliy of suiable hardware and runs in realime (2 minues for mos shown measures), he evaluaion of he mean-field model for a given parameer seing is done in a couple of minues. V. RELATED WORK The noion of mean-field is ofen used in he lieraure, wih differen meanings. The mean-field concep was firs inroduced in physics. I has been used in he conex of Markov chain models of sysems like plasma and dense gases where he srengh of he ineracion beween paricles is inversely proporional o he size of he sysem. A paricle fracion no aware of imesource Fig. 7..9.8.7.6.5.4.3.2 G min = Gmax = 25 2. G min = 5, Gmax = 35 G min = 5, Gmax = 45 2 3 4 5 6 Synchronisaion speed and average hop coun. 9 8 7 6 5 4 3 average hopcoun is seen as under a collecive force generaed by he oher paricles in a coninuous ime and space seing. In he area of communicaion neworks, mean-field convergence resuls have been applied in various forms o a variey of case sudies, including TCP connecions [7], [8], [9], [2], HTTP flows [2], bandwidh sharing [22], ransporaion neworks [23], swarm roboic sysems [24], repuaion deerminaion [5], queueing neworks [25], [26], [27] and Inerne congesion conrol [28]. We are no familiar wih prior work dealing wih meanfield heory for he evaluaion of gossip proocols. Previous work on gossip proocols has used a noion of mean-value and infinie limi (when he number of nodes N ) o simplify compuaion for heir analysis. Noably in [29], Bonne sudied he evoluion of he in-degree disribuion of nodes execuing he Cyclon proocol [7]. The saes of he associaed Markov chain represen he fracion of nodes wih a specific in-degree disribuion. From he designed Markov chain he deermined he disribuion o which he proocol converges. The auhor showed ha he sysem converges by consrucing a generaing funcion, a series whose coefficiens encode he in-degree disribuion. The generaing funcion hen enabled algebraic means o compue he mean value and he sandard deviaion of he saionary disribuion. Sojanovic e al. [3] analysed and compared delay performance of nework coding and cooperaive diversiy in a single-hop wireless nework. The auhors performed an asympoic analysis (for he number of nodes N ) of he expeced delay associaed wih he broadcasing of a file consising of a cerain amoun of packes. VI. CONCLUSION The main moivaion for developing a modelling mehodology for gossip proocols is ha, alhough hese proocols are appealing wih respec o scalabiliy, robusness, and simpliciy, i is hard o quaniaively predic he performance according o a paricular meric or analyse furher possible opimisaions and limiaions analyically. We have demonsraed ha mean-field analysis is suiable for gossip proocols. The following premises enable mean-field analysis: here is a very large number of idenically behaving nodes (symmery propery [3]);
here are no cenral servers or global resources; he behaviour of a single node can be described in a local way; he number of saes of a single node is small in comparison o he number of nodes; ransien measures ( a ime ) are of ineres. Exensions of he heory presened here would also allow for he incorporaion of a global memory, he failure or enering/leaving of nodes [5], he employmen of coninuousime models, and seady-sae measures [32]. However, he mean-field approach does no allow for he evaluaion of a cenrally managed nework, he separae modelling of one single node or he inclusion of opographic informaion on he nework. In his paper we considered wo applicaions of gossip proocols: along wih he presenaion of he necessary heory, we developed a simple informaion disseminaion model. The suiabiliy of he mean-field approximaion mehod was shown by comparing he resuls obained by analyically solving he resuling DTMC, by compuing he mean-field limi, and by simulaing he sysem. As a larger case sudy for an aggregaing gossip proocol we derived a mean-field model for basic GTP. I includes he hop coun meric and he consan gossip delay, and also akes ino accoun enforced updaes due o he expiraion of he sandalone period. We validaed he fi of he mean-field model maching i o emulaion resuls aken from [4]. Then we used he mean-field model o derive a large variey of ineresing measures, also considering dynamically adjused gossip delays. As for fuure work, we plan o invesigae mean-field analysis for alernaive sochasic models for he nodes, e.g., by moving o he coninuous-ime conex [32] or by inroducing non-deerminism using Markov decision processes [33]. REFERENCES [] M. Jelasiy, R. Guerraoui, A.-M. Kermarrec, and M. van Seen, The peer sampling service: Experimenal evaluaion of unsrucured gossipbased implemenaions, in Proc. ACM/IFIP/USENIX Middleware Conf., ser. LNCS, vol. 323. Springer, 24, pp. 79 98. [2] M. Kwiakowska, G. Norman, and D. Parker, Analysis of a gossip proocol in PRISM, SIGMETRICS Perform. Eval. Rev., vol. 36, no. 4, pp. 7 22, 28. [3] R. Bakhshi, L. Cloh, W. Fokkink, and B. R. Haverkor, Mean-field analysis for he evaluaion of gossip proocols, SIGMETRICS Perform. Eval. Rev., vol. 36, no. 3, pp. 3 39, 28. [4] A. Demers, D. Greene, C. Hauser, W. Irish, J. Larson, S. Shenker, H. Surgis, D. Swinehar, and D. Terry, Epidemic algorihms for replicaed daabase mainenance, in Proc. ACM Symp. on Principles of Disribued Compuing. ACM Press, 987, pp. 2. [5] M. Jelasiy, W. Kowalczyk, and M. van Seen, Newscas compuing. Vrije Universiei Amserdam, Tech. Rep. IR-CS-6, 23. [6] D. Gavidia, S. Voulgaris, and M. van Seen, A Gossip-based Disribued News Service for Wireless Mesh Neworks. in Proc. Conf. on Wireless On demand Nework Sys. and Services. IEEE, 26, pp. 59 67. [7] S. Voulgaris, D. Gavidia, and M. van Seen, Cyclon: Inexpensive membership managemen for unsrucured P2P overlays. J. Nework and Sys. Manage., vol. 3, no. 2, pp. 97 27, 25. [8] A. Allavena, A. Demers, and J. Hopcrof, Correcness of a gossip based membership proocol, in Proc. ACM Symp. on Principles of Disribued Compuing. ACM Press, 25, pp. 292 3. [9] Q. Zhang and D. Agrawal, Dynamic probabilisic broadcasing in MANETs. J. of Parallel and Disribued Compuing, vol. 65, no. 2, pp. 22 233, 25. [] M. Jelasiy, A. Monresor, and O. Babaoglu, Gossip-based aggregaion in large dynamic neworks, ACM Trans. Compu. Sys., vol. 23, no. 3, pp. 29 252, 25. [] A.-M. Kermarrec and M. van Seen, Gossiping in disribued sysems, SIGOPS Oper. Sys. Rev., vol. 4, no. 5, pp. 2 7, 27. [2] P. Cosa, V. Gramoli, M. Jelasiy, G. P. Jesi, E. Le Merrer, A. Monresor, and L. Querzoni, Exploring he inerdisciplinary connecions of gossipbased sysems, SIGOPS Oper. Sys. Rev., vol. 4, no. 5, pp. 5 6, 27. [3] K. Iwanicki, M. van Seen, and S. Voulgaris, Gossip-based clock synchronizaion for large decenralized sysems, in Proc. Workshop on Self-Managed Neworks, Sysems and Services, ser. LNCS, vol. 3996. Springer, 26, pp. 28 42. [4] K. Iwanicki, Gossip-based disseminaion of ime, Maser s hesis, Warsaw Universiy and Vrije Universiei Amserdam, 25. [5] J.-Y. L. Boudec, D. McDonald, and J. Mundinger, A generic mean field convergence resul for sysems of ineracing objecs, in Proc. Conf. on he Quaniaive Evaluaion of Sysems. IEEE, 27, pp. 3 8. [6] M. Jelasiy, A. Monresor, G. P. Jesi, and S. Voulgaris, PeerSim: A peer-o-peer simulaor. hp://peersim.sourceforge.ne/. [7] F. Baccelli, D. McDonald, and J. Reynier, A mean-field model for muliple TCP connecions hrough a buffer implemening RED, Perform. Eval., vol. 49, no. -4, pp. 77 97, 22. [8] F. Baccelli, A. Chainreau, D. De Vleeschauwer, and D. McDonald, HTTP urbulence. AMS Neworks and Heerogeneous Media, vol., pp. 4, 26. [9], A mean-field analysis of shor lived ineracing TCP flows, SIGMETRICS Perform. Eval. Rev., vol. 32, no., pp. 343 354, 24. [2] P. Tinnakornsrisuphap and A. Makowski, Limi behavior of ECN/RED gaeways under a large number of TCP flows, in Proc. of IEEE INFOCOM, vol. 2. IEEE, 23, pp. 873 883. [2] F. Baccelli, D. McDonald, and M. Lelarge, Measable regimes for muliplexed TCP flows. in Alleron Conf. on Communicaion, Conrol, and Compuing, 24, pp. 5. [22] S. Kumar and L. Massoulié, Inegraing sreaming and file-ransfer inerne raffic: fluid and diffusion approximaions, Queueing Sys. Theory Appl., vol. 55, no. 4, pp. 95 25, 27. [23] L. Afanassieva, S. Popov, and G. Fayolle, Models for ransporaion neworks. J. of Mah. Sciences, vol. 84, no. 3, pp. 92 3, 997. [24] A. Marinoli, K. Eason, and W. Agassounon, Modeling Swarm Roboic Sysems: A Case Sudy in Collaboraive Disribued Manipulaion, In. Journal of Roboics Research, vol. 23, no. 4, pp. 45 436, 24. [25] D. Dawson, J. Tang, and Y. Zhao, Balancing queues by mean field ineracion, Queueing Sys. Theory & Appl., vol. 49, no. 3 4, pp. 335 36, 25. [26] F. Karpelevich, E. Pechersky, and Y. Suhov, Dobrushin s approach o queueing nework heory. J. of Applied Mahemaics and Sochasic Analysis, vol. 9, no. 4, pp. 373 397, 996. [27] N. Vvedenskaya and Y. Suhov, Dobrushin s mean-field approximaion for a queue wih dynamic rouing. Markov Proc. Rel. Fields, vol. 3, no. 4, pp. 493 526, 997. [28] W. Kang, F. Kelly, N. Lee, and R. Williams, Fluid and Brownian approximaions for an inerne congesion conrol model, in Proc. Conf. on Decision and Conrol, vol. 4, 24, pp. 3938 3943. [29] F. Bonne, Performance analysis of Cyclon, an inexpensive membership managemen for unsrucured P2P overlays. Maser s hesis, ENS Cachan Breagne, Universiy of Rennes, IRISA, 26. [3] I. Sojanovic, M. Sharif, and D. Sarobinski, Daa disseminaion in wireless broadcas channels: Nework coding or cooperaion, in Proc. Conf. on Informaion Sciences and Sysems. IEEE, 27, pp. 265 27. [3] R. Bakhshi, F. Bonne, W. Fokkink, and B. Haverkor, Formal analysis echniques for gossiping proocols, SIGOPS Oper. Sys. Rev., vol. 4, no. 5, pp. 28 36, 27. [32] A. Bobbio, M. Gribaudo, and M. Telek, Analysis of large scale ineracing sysems by mean field mehod, in Proc. Conf. on Quaniaive Evaluaion of Sysems. IEEE, 28, pp. 25 224. [33] M. Puerman, Markov Decision Processes. Wiley, 994.