Skip to content Skip to sidebar Skip to footer
Showing posts with the label Bigdata

Clean One Column From Long And Big Data Set

I am trying to clean only one column from the long and big data sets. The data has 18 columns, more… Read more Clean One Column From Long And Big Data Set

How To Shuffle A Text File On Disk In Python

I am working with a text file of about 12*10^6 rows which is stored on my hard disk. The structure … Read more How To Shuffle A Text File On Disk In Python

Get A List Of Subdirectories

I know I can do this: data = sc.textFile('/hadoop_foo/a') data.count() 240 data = sc.textFi… Read more Get A List Of Subdirectories