• LFTP - some commands and references

    Great little FTP client, but a steeper learning curve from normal GUI based ones. Here are a few commands/references I occasionally use. I’ll try to keep this post up to date as I go.

    Connect to server:

    (If wanting to connect via SFTP, add it at the start like below)

    lftp
    open sftp://host
    user USER
    

    It should then ask for a password. You can then ls and you should see the files.

    Download a directory:

    (A mirror command downloads multiple files at once and uses segmentation:)

    queue mirror -P 5 --use-pget-n=5 Directory
    

    One file

    pget -n 5 File.dmg
    

    Queue

    To delete a transfer from your queue, add the -d switch:

    queue -d #
    

    To stop and delete a running transfer, use the kill command.

    If you set lftp to allow multiple transfers simultaneously, kill #, where # is the number listed by jobs.

    kill all will stop and delete all transfers.

    Passwords

    If you want your password to be saved when you create a bookmark, use the command:

    set bmk:save-passwords true
    

    Sample aliases

    Edit MAC OS X at: /usr/local/etc/lftp.conf

    ##MyMods 
    set bmk:save-passwords true 
    alias dir !ls -Fsh --group-directories-first
    

    And a list of aliases:

    alias j jobs
    alias jv jobs -v
    alias q queue
    alias qs queue stop
    alias k kill
    alias ka kill all
    alias m mirror -c -P 2 --use-pget-n=5
    alias p pget -c -n 6
    

    Adding bookmarks

    To do this, use the “bookmark” command.

    Then these:

    add <name> [<loc>] 
    del <name> 
    list
    

    Full example of the process

    1. Navigate in Terminal to the folder you want, then connect to FTP server from that terminal window
    2. After it connects, it should say cd ok
    3. List all files with ls
    4. To download a file from the FTP, type: q p FILENAME
    5. NOTE: You don’t have to type the whole file name. Just the start, then you can fill the rest by pushing the tab key.

    It’s an easy way to use the FTP server to upload and download files.

  • The Martian

    I finished reading Andy Weir’s The Martian last night.

    I started the day before. I love Mars!

    It’s an easy read. Check it out.

  • Moving from Wordpress to GitHub Pages

    Bye bye Wordpress. Hello GitHub Pages!

    I just decided to move my blogging platform away from the tried-and-true Wordpress to GitHub.

    Why?

    Well, for a few reasons.

    • I can write posts in Markdown
    • I don’t have to worry about zero-day Wordpress vulnerabilities
    • Version tracking of blog posts!
    • Why not?

    It wasn’t as hard as I thought it would be.

    I followed this guide, which was really helpful in showing how to export comments, move them to Disqus, etc.

    Now I’m just waiting for the DNS changes to take place, and it’ll be all done (apart from learning the way Jekyll works!)

  • Deleting Amazon AWS Glacier vaults and archives

    I once used Arq – a backup program for Mac – to backup all my documents to Amazon’s AWS cloud. I decided to use Glacier, a low-cost and long-term option to do this.

    However, it all got out of sync and Arq wanted to re-back everything up when I reinstalled my Mac OS X, so I decided to ditch it.

    That left me with many vaults on Amazon Glacier that I couldn’t get rid of – but was still paying for. The AWS Console only allows you to delete vaults if they are empty.

    There’s this error message. How annoying!

    Screen Shot 2014-06-25 at 11.01.08 am

    There is a way to delete the archives and vaults, but it takes a few steps. Here’s how I did it.

    1) Find a VPS provider and spin up a new server

    I used Digital Ocean. It’s quick and cheap and easy. I used a VPS because at the end, I can nuke it and not worry about having programs I will no longer need installed on my computer.

    Screen Shot 2014-06-25 at 10.34.48 am

    I installed Ubuntu 12.04.4 x64

    2) SSH into your new VPS

    Once you’ve SSHed into the server, you’ll have to install a bit of software.

    I used this “Perl Multithreaded multipart sync to Amazon Glacier service.

    To install, I ran this commands:

    sudo apt-get update
    sudo apt-get install software-properties-common python-software-properties
    sudo add-apt-repository ppa:vsespb/mt-aws-glacier
    sudo apt-get update
    sudo apt-get install libapp-mtaws-perl
    

    That will install the software needed to interface with Glacier.

    From there, you’ll need to get your Amazon security tokens to the software can connect to the cloud.

    You’ll need to put it into a config file. Here’s an example of what it should look like:

    key=YOURKEY
    secret=YOURSECRET
    #region: eu-west-1, us-east-1 etc
    region=us-east-1
    #protocol=http (default) or https
    protocol=http
    

    I called my file: glacier.cfg

    Once you’ve done that, you’ll need to get the “inventory” of each vault you want to delete.

    To do that, log into the AWS Console and go to the Glacier section. Find the region the vault is stored.

    You’ll see a list of the vaults:

    Screen Shot 2014-06-25 at 10.51.21 am

    Then run this command for each of the vaults:

    mtglacier retrieve-inventory --config glacier.cfg --vault vault-name
    

    Replace “vault-name” with the name of the vault you see in the list.

    If you’ve done it correctly, you should see something like:

    MT-AWS-Glacier, Copyright 2012-2014 Victor Efimov http://mt-aws.com/ Version 1.115
    
    
    PID 2551 Started worker  
    PID 2551 Retrieved Inventory, job id XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX  
    OK DONE
    

    Do it for each of the vaults you want to delete.

    Once you’ve done that, you’ll need to wait a while. Possibly a few hours. I waited about eight or so.

    After you’ve waited, there are a few more commands to run:

    Firstly, you’ll need to download the inventory. Replace “vault-name” twice. Once for the name of the vault and once so the vault’s name matches the log file.

    mtglacier download-inventory --config glacier.cfg --vault vault-name --new-journal vault-name.log
    

    You’ll see a similar message as above.

    Then, you need to purge the vault. Here’s the command. Don’t forget to replace the “vault-name” and make sure the .log file matches what you set it to in the previous step.

    mtglacier purge-vault --config glacier.cfg --vault vault-name --journal vault-name.log
    

    It should then run through the vault and delete the archives. Depending on how many archives are in the vault, it may take a while. It took me about 10-15 minutes to delete 20,000 archives – so don’t get impatient. Just let it run.

    Once that’s done, you may have to wait a while again. Then the vaults will be empty. You can then delete from the AWS Glacier Console!

  • Codefellas: NSA satire

    Codefellas

    This is really cool, or as John Birmingham puts it over at cheeseburgergothic.com:

    Dunno if anyone but me has been watching Wired’s Codefellas series, but it’s one of the sharpest little blipverts of modern satire I’ve come across of late.

    It’s basically an updated Odd Couple meme, set in the NSA, with Oscar and Felix swapped out for Topple, an ageing Cold Warrior and Nicole, a code hacker from Special Projects.

    So, here are the links:

    Ep1 (When Topple met Winters) http://www.youtube.com/watch?v=aHz7iYMqSZQ

    Ep2 (Meet Big Data) http://www.youtube.com/watch?v=RA_nM1k5B2g

    Ep3 (How to Hack a Website) http://www.youtube.com/watch?v=WW6GE-kmP64

    Ep4 (The AntiSocial Network)
    http://www.youtube.com/watch?v=M3gKCMSgZ4k

    Ep5 (Spy vs. Spy) http://www.youtube.com/watch?v=ogJbS84ZhQY

    Ep6 (Blackmail) http://www.youtube.com/watch?v=BTMYBmesKn8

    Ep7 (Reasons the NSA Should Hire Buzzfeed Staffers) http://www.youtube.com/watch?v=4t-tDNhP10Q