Skip to content Skip to sidebar Skip to footer
Showing posts with the label Hadoop

Why "importerror: No Module Named Builtins" Appears After Importing Hive From Pyhive Package?

I have a simple question to ask. I have been trying to execute HIVE queries from Python using impyl… Read more Why "importerror: No Module Named Builtins" Appears After Importing Hive From Pyhive Package?

Spark 2.2 Thrift Server Error On Dataframe Numberformatexception When Query Hive Table

I have Hortonworks HDP 2.6.3 running Spark2 (v2.2). My test case is very simple: Create a Hive tab… Read more Spark 2.2 Thrift Server Error On Dataframe Numberformatexception When Query Hive Table

Unpickle File From Hdfs

I'm currently using Python 3 and would like to load a pickle file out of HDFS. from pywebhdfs.… Read more Unpickle File From Hdfs

Python Udfs In Pig

I've seen the documentatio here, but I confess that I feel it rather lacking. I was wondering … Read more Python Udfs In Pig

Spark Java.lang.verifyerror

I get the following error when I try to call I use python client for the spark. lines = sc.textFil… Read more Spark Java.lang.verifyerror

Get A List Of Subdirectories

I know I can do this: data = sc.textFile('/hadoop_foo/a') data.count() 240 data = sc.textFi… Read more Get A List Of Subdirectories

Pyspark --py-files Doesn't Work

I use this as document suggests http://spark.apache.org/docs/1.1.1/submitting-applications.html sp… Read more Pyspark --py-files Doesn't Work

Hive Parseexception In Drop Table Statement

I'm using python and pyodbc module in particular to execute Hive queries on Hadoop. The portion… Read more Hive Parseexception In Drop Table Statement

Create Custom Writable Key/value Type In Python For Hadoop Map Reduce?

I have worked on Hadoop MR for quite some time and I have created and used custom(extension) Writab… Read more Create Custom Writable Key/value Type In Python For Hadoop Map Reduce?

How To Decode .lzo_deflat File?

does anyone have either a command-line or python library that can decode a .lzo_deflate file? Thes… Read more How To Decode .lzo_deflat File?

Python Write To Hdfs File

What is the best way to create/write/update a file in remote HDFS from local python script? I am a… Read more Python Write To Hdfs File

Get List Of Files From Hdfs (hadoop) Directory Using Python Script

How to get a list of files from hdfs (hadoop) directory using python script? I have tried with foll… Read more Get List Of Files From Hdfs (hadoop) Directory Using Python Script