Thursday, May 15, 2014

Nice linux command to ignore lots of files

I've been working with git from command line lately and in this particular project we have a folder to store cache files (we use other cache stores in production) which gets crowded really quickly and doing a git status was not pleasant at all, here is a screenshot from inside that folder (file/cache):
at the end of the list it would show the files I was interested in but still... A couple of minutes later I came up with this one liner to remove all of these files:


git status | egrep cache | tr -s ' ' | cut -d ' ' -f 2 | xargs git rm --cached $1;

Let me explain why it works, the first part (git status) runs the normal git status command but this one is cluttered with extra information, I just want the files that belong to the cache folder so a simple egrep cache filters the lines that do not contain the word "cache". Now, I wasn't sure if those were spaces or tab characters so to make it more universal I decided to remove duplicated spaces with tr -s ' ', it does not remove all spaces, just the duplicated ones so '    ' becomes ' '. The next part is the cut -d ' ' -f 2 which basically extracts the information from column 2, and finally pass that to git rm --cached 

Note that all of these commands are "glued" with the pipe character ( this one: | ) which takes the output from the previous command and passes it as a param to the next command.

Thursday, May 1, 2014

Password Cracking with Hashcat

Disclaimer: Anything posted here is in the spirits of education, and education only. I am not responsible for what you do with the information here posted.

Ever since I attended Siren's talk on DDoS (she totally rocks btw!) I got -once again- interested in security and today I undusted an old proyect I worked on and tried to log in, no luck try after try until eventually I thought "well I can of course just reset it... or have some fun and crack it", after all, it uses md5 and its been a while since md5 was first cracked... shouldn't be too hard to find my old password right? well sure there are tools but the processing power available to me still makes it an arduous task... to make things worse the hashing format isn't plain md5, its a triple md5-d password, so the process is not straightforward.

Enter Hashcat, a very robust tool to crack passwords, their documentation wasn't dummy-proof and even after reading examples, their wiki and the help command I couldn't get anything working, fortunately "Xanadrel" in IRC helped me through and shortly after I got a better understanding of how it works. So, fair warning, I am not a cracker, Im not a super smart guy, I couldn't even get this to work on my own in the first try. What Im sharing here is what I learned today.

Password cracking is slow...

Im doing this in a virtual machine with 4 cores available and 3.3Ghz, my original password would take 16hrs assuming Im right and the last 2 characters are digits. Im sure putting more cores to work would highly improve the processing time but Im not going to do that.

Mask

The mask tells the format of the original password and is great if we know something about the password we are trying to crack like the length and the type of characters in the password. If we know that in a certain position of the password there is a number, we can tell Hashcat about it and the time it takes to find the password is reduced. Here is a nice table with the replacements:

What we knowReplace with
its a number?d
its a upper case letter?u
its an lower case letter?l
its a symbol like <space>!"#$%&'()*+,-./:;<=>?@[\]^_`{|}~ ?s
any of the cases above?a

Here are some examples of masks for specific passwords:
PasswordMask
14705?d?d?d?d?d
azctqq?l?l?l?l?l?l
1s5f7u?d?l?d?l?d?l
! 4L?s?s?d?u

Attack Types

Since we want to make this right, and we know something about the password, we can define a specific type of attack, this is the algorithm that Hashcat will use to find the password. Since we know something about the original password we can use the mask and because of this we need to specify a "brute force" attack, in reality this isn't a brute force attack but we need to define it as such (this was explained to me by Xanadrel). 
Depending on your operating system the way to pass params varies, since Im in linux this is how we pass the param for brute force:

-a 3

I believe in windows you need to use the long format.

Hash Types

There are many combinations to using md5, they call this "hash types", in this case since we know the password is a triple md5-d hash, we can find that in the references (run hashcat with the param --help and find the "References" section) with the number 3500 so we would add this param to the equation:

-m 3500

Increment

Hashcat assumes that if we set a mask of 7 characters we also want to check for passwords with 6 characters and less, if we know the exact length or even an approximate length it will be very useful to tell hc about it, we do this with the pm-min and pm-max params:

--pm-min=6 --pm-max=7

The Recipe

Hashcat reads hashes from a file, so make sure to put your hash in a file, it doesnt need anything special or fancy, just a one line with your hash and you're good to go, lets say you named this file "to_crack.txt", the final command would look like this:

shell> ./hashcat-cli64.bin -m 3500 -a 3 -show --pw-min=7 --pw-max=7 to_crack.txt ?a?a?a?a?a?d?d


I didn't need any salts but if you do, you add them to the end of your hash in your hash file so it would look like this:


ed1791de507c63335e735bd6ce7cd7bb:salt

The format is:
<hash>:<salt>

(One per line)

The output isn't all bells and whistles so you might miss where it says that it found your password, just look for the message "All hashes have been recovered" and above it you will find the hash and the password.

So there you have it, hope this helps you understand a bit how this magic tool works. If you have any improvements or comments in general feel free to post in the comments.

mysqldump: table doesn't exist

Today I needed to dump a database, nothing special, until mysql dump growled a "table <x> doesn't exist when using LOCK TABLES", ran a check
~$ mysqlcheck -udb_user -p database_name
but everything showed up fine... then I thought "well lets not lock it...", so this worked for me:
~$ mysqldump --skip-comments --add-drop-table --skip-lock-tables --user=my_user --password=my_password database_name >> db.sql
This database in particular was imported by copying files, I fixed the permissions and owner and it is working fine otherwise so I don't know what could be causing this problem which is why I dont count this as a "fix" but just a workaround, wiser folks may understand better what is happening and enlighten me.