How to Manage Big Passwordlists

In the penetration testing journey we must learn how to brute force attack. But the success of brute force attack depends on our password list. We need a bigger and specialized password list to be successful in dictionary attack or brute force attack. We can find over 20GB password lists on the internet easily.

Managing a bigger sized password list is always a headache for everyone. Ever tried to edit an <10GB wordlist text file on a mid-range laptop? After trying this anyone will know what is real pain!

But sometimes we need to create a custom wordlist and we also want to use the big sized common password list. Sometimes we also need to delete some passwords from the list or add some passwords to the list. In this situation we have two ways to do it.
how to manage big passwordlists

First is we may use a famous tool called Dymerge. This tool might be famous but we think it sucks. Then we should follow the second one. Wait a bit first we are going to learn about Dymerge tool then we learn about the second method.

Dymerge in Kali Linux


Dymerge is a python tool that we can use our password lists. It is a simple command line tool that help us to marge wordlists or add/remove specific keywords from a wordlist.

 Let's use Dymerge in our Kali Linux system. First we need to copy it from it's GitHub repository by using following git clone command:
git clone https://github.com/k4m4/dymerge
The screenshot of the command is following:
cloning Dymerge in Kali Linux

Then we need to navigate inside the dymerge directory by using following command:
cd dymerge
We can see following files in dymerge directory by using ls command:
ls
In the following screenshot we can see the dymerge.py the python script we need.
dymerge python tool
To see Dymerge's help option we can use following command:
python dymerge.py --help
Dymerge will open the help menu after the above command as the following screenshot:
dymerge help option
Here we can see that we can marge two wordlists using this tool. For example we gathered some wordlist files on our Desktop.
wordlist text files on our desktop
Now we will marge all these wordlists into a giant wordlist and we want the output in compressed format. Here we compress all the wordlists on the directory, for that we just need to specify the directory only.

If we want to merge individual wordlists then we have to specify the file names also. -o flag is used for the output location & format. We also want to insert one more password "KaliLinuxIn" into our giant wordlist, so use need to use -i flag.

We can remove the duplicate passwords and keep the unique passwords only by just using the -u flag. We also can reverse the passwords by using -r flag but we are not using that on our this example.

So we apply the following command in our terminal.
python dymerge.py /home/kali/Desktop/my-wordlists -u -f -o /home/kali/Desktop/my-wordlistsgiantwordlist.txt -i KaliLinuxIn -z bz2
Here we have run the command and the output is following:
dymerge is running
In the above screenshot we can see Dymerge is started its job. We just need wait some seconds (It's depends on our wordlist's size and our machine's processing power). After finish the work it looks like the following screenshot:
dymerge is finished it's job
We had choose the same directory for the giant wordlist's output as zip. After complete the process we got the giant wordlist on our required directory or folder, as we can see in the following screenshot.
our giant wordlist is created
We got the giant zipped wordlist. Don't follow our wallpaper it changes automatically in minutes
That is how Dymerge works. But this can be replaced by some simple bash commands that is the second method. Bash can easily replace all it's functions, so why we need this? That why we told that Dymerge sucks.

DyMerge Vs Bash


When Bash commands can replace Dymerge then why we need to carry an extra python script on our machine. We don't need to download and run an extra tool to do it.

Moreover, Dymerge doesn't work well with too large files. This is because it loads every word from each dictionary into an array, causing the computer system's memory to overload and possibly even crash.

Here is some Dymerge uses and Bash commands that can replace those:

Dymerge command:
python dymerge.py /usr/share/wordlists/rockyou.txt /lists/cewl.txt -s -u
Bash command to replace this
sort -u /usr/share/wordlists/rockyou.txt /lists/cewl.txt > output.txt

Dymerge command:
python dymerge.py /lists/cewl.txt /lists/awlg.txt -s -u -i Passw0rd,inpuT
Bash command to replace this
sort -u /lists/cewl.txt /lists/awlg.txt <(echo 'Passw0rd\ninpuT') > output.txt

Dymerge command:
python dymerge.py ~/fsocity.dic -s -u -r -o ~/clean.txt
Bash command to replace this
sort -r <~/fsocity.dic | uniq> clean.txt

Dymerge command:
python dymerge.py /dicts/crunch.txt /dicts/john.txt -s -u -f -z bz2
Bash command to replace this
sort -u /dicts/crunch.txt /dicts/john.txt | bzip2 > output.bz2
Even though all of DyMerge's main features can be fulfilled with classic bash, the tool has actually gained publicity.  -- Nikolaos Kamarinakis (Creator of DyMerge)
So, this all for this tutorial hope everyone liked this. In this tutorial we have learned how to manage bigg wordlists on our system. For anything commend down we always reply. Also don't forgot to follow us on Twitter and Medium.





No comments:

Please do not spam here. It is comment box not a spambox. Promotional links are not allowed.

Powered by Blogger.