paper critique

Paper: Result Verification and Trust-based scheduling in Open Peer-to-peer Cycle Sharing Systems.

Summary + Liked:
Paper presents interesting trade-offs like overhead in turnaround time (Quiz) vs. overhead of compute resources (duplication in replication) vs. desired level of accuracy.
Paper extends the verification schemes to P2P model. If one can certain honest task execution in such environment then same solution can be used in client-server kind of models as well.
Paper also characterizes different kinds of reputation system aggregation. These different approaches of aggregation present trade off between reputation system performance and cost of increased network traffic and communication overhead.
Notion of accuracy on demand follows Lampson’s hint of “leave it to client” where client have freedom to the various result verification parameters to match desired level of accuracy or overhead.
With the advent of BOTNETS in P2P systems, chances of cheating by collusion increases in P2P system hence quiz based verification scheme can play an important role there.
Quiz based approach can also be used to keep reputation mechanism more reliable as results returned by quiz is more certain then replication and can be used to feed into statistical model used in calculating reputation of nodes.


Disliked:
One improvement to the paper can be “Use of hybrid approach of quiz and replication” in conjunction of adaptive reputation system to achieve desired level of accuracy at minimum overhead possible. As we can see Quiz based verification leads to extra compute time while replication does not ensures 100% accuracy hence using a hybrid approach can lead to reduction in quiz ratio as well as replication factor which enables faster results and less overhead in terms of compute time. The decision of quiz ratio and replication factor can be function of resource availability, compute nodes reputation, desired level of accuracy and time of execution.
It seems no actual implementation has been done and work is more theoretical. For instance “Accuracy on demand” and “Embedding quiz in the task” has not been implemented in the practice by the time paper was published.
The scenarios where required number of quizzes exceeds the configured package size should be dealt in a more intelligent way rather than sending the whole package full of quizzes. Use of replication in conjunction to quiz can play an important role here.