Hadoop FS Shell Commands

Hadoop File System commands – start with Hadoop FS.
I want to put couple of local directory files into hdfs
FS Shell Guide

ls –  listing of what’s in my home directory on the Hadoop cluster
Usage: hadoop fs -ls <args>
For a file returns stat on the file with the following format:
filename <number of replicas> filesize modification_date modification_time permissions userid groupid

hadoop fs –ls

LS Command
Put – takes a local file(s) and places it into hdfs
Usage: hadoop fs -put <localsrc> … <dst>
upload our purchases.txt file, it takes a few seconds to upload.

hadoop fs –put purchases.txt
 hadood fs –ls

see that that file is now in hdfs.
tail – Displays last kilobyte (few lines) of the file
Usage: hadoop fs -tail [-f] URI

hadoop fs -tail purchases.txt

Tail
cat – display the entire contents of the file
Usage: hadoop fs -cat URI [URI …]

hadoop fs – cat purchases.txt

mv – rename the file, i.e moves files from source to destination
Usage: hadoop fs -mv URI [URI …] <dest>

hadoop fs –mv purchases.txt newname.txt

get – Copy files to the local file system
Usage: hadoop fs -get [-ignorecrc] [-crc] <src> <localdst>

hadoop fs -get jobout mylocal.txt/user/hadoop/file localfile

rm – delete a file
Usage: hadoop fs -rm URI [URI …]

hadoop fs –rm newnametxt  -  get rid of newname.txt from hdfs.

mkdir – create a directory in hdfs
Usage: hadoop fs -mkdir <paths>

hadoop fs –mkdir myinput

Upload purchases.txt and place it in the myinput directory.

hadoop fs –put purchases.txt myinput

mkdir

Advertisement

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: