IFMapReduce: Interactive analysis of Big Data
Authors
Jensen, Christian Møller ; Larsen, Lars Harald
Term
4. term
Education
Publication year
2015
Submitted on
2015-06-08
Pages
43
Abstract
Rapporten undersøger mulighederne for interaktiv analyse af Big Data (meget store datamængder) på .NET-platformen via F# Interactive, et interaktivt miljø til at køre F#-kode med det samme. Undersøgelsen omfatter designet af et programmeringsrammeværk, IFMapReduce, som udnytter F# Code Quotations (en teknik til at repræsentere kode som data) til at sende kode ud til maskiner i en beregningsklynge (cluster computing). En prototype af IFMapReduce er implementeret og evalueret gennem benchmarktests med PageRank- og WordCount-algoritmerne. Rapporten præsenterer også flere forslag til, hvordan IFMapReduce og prototypen kan forbedres.
This report explores how to do interactive analysis of Big Data (very large datasets) on the .NET platform using F# Interactive, an environment for running F# code immediately. It includes the design of a programming framework called IFMapReduce that uses F# Code Quotations (a way to represent code as data) to enable shipping code to machines in a computing cluster. A prototype of IFMapReduce was implemented and evaluated through benchmarks using the PageRank and WordCount algorithms. The report also presents several suggestions for improving IFMapReduce and its prototype.
[This abstract was generated with the help of AI]
Keywords
Big Data ; MapReduce ; Spark ; FMapReduce ; IFMapReduce ; Fsharp ; F# ; Fsharp interactive ; IFDataset ; Code Quotations
Documents