message propagation in a network, p2p approach, part 2

The more i write or think about it, the more i know, what i would like to have:

• There is a bytestream out there. This bytestream is identified by a unique hash.
  
• Peers are ready to seed this bytestream to other peers.
  
• Other peers have a way to find out, what hashes are available, hashes of their interest (which would group peers together)
  
• Other peers have a way to find out, which peers seed the bytestream identified by the hash or which peers have parts of it
  

Lets transform it to the message propagation example:

• A peer has a message queue as a resource. It writes the messages into the queue. The queue itself has an underlying bytestream, which itself could be a simple file it writes into.
  
  
• The peer writes properties about this resource into a file with a certain format and announces it somewhere (the torrent/tracker equivalent).
  
  
• Another peer knows where to look for "torrents" and finds the ones of its own interest. Then it asks someone else, who offers this resource at the moment and starts to leech from it (or lets say: it replicates that resource)
  
  
• On that replicated resource (which can be a file again) sits a bytestream, which is again the source for the message queue, which would extracts the messages from it.
  

Checking, yes, it still sounds understandable to me.

No thoughts about recovery are needed yet!

If a peer crashes, its bytestream can no more be continued, sure. But other peer might have replicated this resource right before its crash, so other peers can still leech it. There might be archiver peers in the network which do nothing else than replicating certain resources. Peers might chose as well to use only the archiver peers to leech from.

And if a crashed peer comes back, it also can recover itself by leeching lost pieces from the other peers and then starts continuing the resource on its own again. Or it creates a new file and abandones the old file.

However, this still sounds a lot like bittorrent.


Okay, summing up shortly:

• PeerA Message Queue writes messages into bytestream BS_A (business logic)
  
• PeerB Message Queue writes messages into bytestream BS_B (business logic)
  
• PeerA replicates BS_B (replication logic)
  
• PeerB replicates BS_A (replication logic)
  
• PeerB Message Queue reads messages from PeerA by reading BS_A (business logic)
  
• PeerA Message Queue reads messages from PeerB by reading BS_B (business logic)
  

I love it already.

There is one missing piece yet. How to represent a bytestream, which has no start and no end. This will be in the next blog entry. I hope.