Live Chat!
using curl to access ftp server
October 6th, 2007 mysurface Posted in Advance, Network, curl | Hits: 39426 |
The title doesn’t sound interesting if you have no idea what is curl. Why we need to use curl to access ftp server, if we can access ftp with tools like ftp in console or gFTP?
Well gFTP is a very handy ftp client with gtk front end, as I use it daily to maintain my files in my web servers. But sometimes we need a command that allows us to put into script, then gFTP is not suitable for that. And default ftp command surprise me that we cannot do things inline. Let say, I wanna download a file from a ftp server by passing the username and password within one line of command, so I can put into my script. I can’t do this with default ftp command!
Curl provides you a way to access ftp server and download, upload files, listing directories and file, and you can write your routine into a script using curl.
Lets look at how we can do it with curl.
The simplest way to access a ftp server with username and password
curl ftp://myftpsite.com --user myname:mypassword With the command line above, curl will try to connect to the ftp server and list all the directories and files in the ftp home directory.
To download a file from ftp server
curl ftp://myftpsite.com/mp3/mozart_piano_sonata.zip --user myname:mypassword -o mozart_piano_sonata.zipTo upload a file to ftp server
curl -T koc_dance.mp3 ftp://myftpsite.com/mp3/ --user myname:mypasswordTo list files in sub directories.
curl ftp://myftpsite.com/mp3/ --user myname:mypasswordList only directories, silent the curl progress bar, and use grep to filter
curl ftp://myftpsite.com --user myname:mypassword -s | grep ^dRemove files from ftp server.
This is a bit tricky, because curl do not support that by default, well anyway, you can make use of -X and pass in the REAL FTP command.
(Check out a list of FTP service Command in rfc 959, under 4.1.3. FTP SERVICE COMMANDS)
curl ftp://myftpsite.com/ -X 'DELE mp3/koc_dance.mp3' --user myname:mypasswordCaution: Make sure you are know what are you deleting! It will not prompts ‘Are you sure?’ confirmation.
Check out curl manual for more,
man curlor this, ^^ (contains different info)
curl --manual | less 
October 8th, 2007 at 11:19 am
You might want to look into use a .netrc file to handle your FTP credentials. It makes things much easier when you need to use FTP on the fly or in bash scripts.
see: http://man.linuxquestions.org/index.php?query=netrc&type=2§ion=5
(the .netrc file must have permissions of 600 to function properly, but it will warn you if you forget)
You can pass your ftp commands to ftp via a text file if you like as in:
ftp < textfile
October 8th, 2007 at 4:01 pm
Great! Thanks Techfun
November 17th, 2007 at 10:54 am
[...] have wrote a post regarding on how to access ftp host using curl. And this time, let us look at how to mount the ftp host to a local directory on top of [...]
February 1st, 2008 at 1:57 am
expanding on Techfun’s post you can also set macros in the .netrc file. Then it can be executed in a shell script with the command
echo “$ macroname” | ftp ftplocation
information on setting up a macro can be found here
http://www.mavetju.org/unix/netrc.php
works well for automated offsite backups
June 29th, 2009 at 6:04 am
I wish to fetch a file from an ftp server via an http proxy, probably squid. Can’t do this with normal ftp, and need to know how to setup the environment so that curl does this. I want to run a script that needs to download some ftp files.
July 25th, 2009 at 3:59 am
Thanks! This is exactly what I was looking for.
As relevant in Aug09 as when it was first posted in Oct07.
December 18th, 2009 at 12:09 am
to do it over proxy
curl -v -x proxy-server:8080 -U proxyuser:proxypassword –user ftpuser:ftpassword ftp://ftp.server.com