vendredi 18 septembre 2015

Spark: After CollectAsMap() or Collect(), every entry has same value

I need to read a text file and change this file to Map. When I make JavaPairRDD, it works well.
However, when I change JavaPairRDD to Map, every entry has same value, more specifically the last data in text file.

Input Text File:

A: ABCDE
B: BCDEF
C: CDEFG

When I read a text file, I used Hadoop custom input format.
Using this format, Key is offset and Value is custom class.

JavaPairRDD:

[0, (A,ABCDE)]
[1, (B,BCDEF)]
[2, (C,CDEFG)]

However, when I do new HashMap<>(JavaPairRDD.collectAsMap()):

[0, (C,CDEFG)]
[1, (C,CDEFG)]
[2, (C,CDEFG)]

I don't know why it happens..
Please help me....



from Newest questions tagged java - Stack Overflow http://ift.tt/1UZuAmX
via IFTTT

Aucun commentaire:

Enregistrer un commentaire