Friday, February 01, 2013
Reducing MapReduce
Vincent Granville on What MapReduce can't do. " ... We discuss here a large class of big data problems where MapReduce can't be used - not in a straightforward way at least - and we propose a rather simple analytic, statistical solution. ... MapReduce is a technique that splits big data sets into many smaller ones, process each small data set separately (but simultaneously) on different servers or computers, then gather and aggregate the results of all the sub-processes to produce the final answer. Such a distributed architecture allows you to process big data sets 1,000 times faster than traditional (non-distributed) designs, if you use 1,000 severs and split the main process into 1,000 sub-processes..... "
Labels:
Architecture,
Map-Reduce
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment