Unified blackhole file with base extensions

Take Note!

With the exception of issues and PRs regarding changes to blackhole/data/boss-net/blackhole, all other issues regarding the content of the produced blackhole files should be made with the appropriate data source that contributed the content in question. The contact information for all of the data sources can be found in the blackhole/data/ directory.

Unified blackhole file with base extensions

This repository consolidates several reputable blackhole files, and merges them into a unified blackhole file with duplicates removed. A variety of tailored blackhole files are provided.

Therefore this repository is a blackhole file aggregator.

Last updated: October 09 2024.
Here's the raw blackhole file Unified blackhole file with base extensions containing 115,497 entries.

List of all blackhole file variants

This repository offers 31 different host file variants, in addition to the base variant, with and without the unified blackhole included.

The Non GitHub mirror is the link to use for some blackhole file managers like Blackholeman for Windows that don't work with GitHub download links.

Host file recipe	Readme	Raw blackhole	Unique domains	Non GitHub mirror
Unified blackhole = (adware + malware)	Readme	link	115,497	link
Unified blackhole + fakenews	Readme	link	117,691	link
fakenews	Readme	link	2,194	link
Unified blackhole + gambling	Readme	link	123,962	link
gambling	Readme	link	8,475	link
Unified blackhole + porn	Readme	link	177,854	link
porn	Readme	link	62,933	link
Unified blackhole + social	Readme	link	118,690	link
social	Readme	link	3,222	link
Unified blackhole + fakenews + gambling	Readme	link	126,156	link
fakenews + gambling	Readme	link	10,669	link
Unified blackhole + fakenews + porn	Readme	link	180,048	link
fakenews + porn	Readme	link	65,127	link
Unified blackhole + fakenews + social	Readme	link	120,884	link
fakenews + social	Readme	link	5,416	link
Unified blackhole + gambling + porn	Readme	link	186,319	link
gambling + porn	Readme	link	71,408	link
Unified blackhole + gambling + social	Readme	link	127,155	link
gambling + social	Readme	link	11,697	link
Unified blackhole + porn + social	Readme	link	181,046	link
porn + social	Readme	link	66,154	link
Unified blackhole + fakenews + gambling + porn	Readme	link	188,513	link
fakenews + gambling + porn	Readme	link	73,602	link
Unified blackhole + fakenews + gambling + social	Readme	link	129,349	link
fakenews + gambling + social	Readme	link	13,891	link
Unified blackhole + fakenews + porn + social	Readme	link	183,240	link
fakenews + porn + social	Readme	link	68,348	link
Unified blackhole + gambling + porn + social	Readme	link	189,511	link
gambling + porn + social	Readme	link	74,629	link
Unified blackhole + fakenews + gambling + porn + social	Readme	link	191,705	link
fakenews + gambling + porn + social	Readme	link	76,823	link

Expectation: These unified blackhole files should serve all devices, regardless of OS.

Sources of blackhole data unified in this variant

Updated blackhole files from the following locations are always unified and included:

Host file source	Home page	Raw blackhole	License	Issues	Description
AdAway	link	raw	CC BY 3.0	issues	AdAway is an open source ad blocker for Android using the hosts file.
add.2o7Net	link	raw	MIT	issues	2o7Net tracking sites based on hostsfile.org content.
add.Dead	link	raw	MIT	issues	Dead sites based on hostsfile.org content.
add.Risk	link	raw	MIT	issues	Risk content sites based on hostsfile.org content.
add.Spam	link	raw	MIT	issues	Spam sites based on hostsfile.org content.
Mitchell Krog's - Badd Boyz Hosts	link	raw	MIT	issues	Sketchy domains and Bad Referrers from my Nginx and Apache Bad Bot and Spam Referrer Blockers
hostsVN	link	raw	MIT	issues	Hosts block ads of Vietnamese
KADhosts	link	raw	CC BY-SA 4.0	issues	Fraud/adware/scam websites.
MetaMask eth-phishing-detect	link	raw	DON'T BE A DICK PUBLIC LICENSE	issues	Phishing domains targeting Ethereum users.
minecraft-hosts	link	raw	CC0-1.0	issues	Minecraft related tracker hosts
MVPS hosts file	link	raw	CC BY-NC-SA 4.0	issues	The purpose of this site is to provide the user with a high quality custom HOSTS file.
shady-hosts	link	raw	CC0-1.0	issues	Analytics, ad, and activity monitoring hosts
Dan Pollock – someonewhocares	link	raw	non-commercial with attribution	issues	How to make the internet not suck (as much).
KhulnaSoft's ad-hoc list	link	raw	MIT	issues	Additional sketch domains as I come across them.
Tiuxo hostlist - ads	link	raw	CC BY 4.0	issues	Categorized hosts files for DNS based content blocking
UncheckyAds	link	raw	MIT	issues	Windows installers ads sources sites based on https://unchecky.com/ content.
URLHaus	link	raw	CC0	issues	A project from abuse.ch with the goal of sharing malicious URLs.
yoyo.org	link	raw		issues	Blocking with ad server and tracking server hostnames.

Extensions

The unified blackhole file is optionally extensible. Extensions are used to include domains by category. Currently, we offer the following categories: fakenews, social, gambling, and porn.

Extensions are optional, and can be combined in various ways with the base blackhole file. The combined products are stored in the alternates folder.

Data for extensions are stored in the extensions folder. You manage extensions by curating this folder tree, where you will find the data for fakenews, social, gambling, and porn extension data that we maintain and provide for you.

Generate your own unified blackhole file

You have three options to generate your own blackhole file. You can use our container image, build your own image, or do it in your own environment. Option #1 is easiest if you have Linux with Docker installed.

Option 1: Use our container image (Linux only)

This will replace your /etc/blackhole.

We assume you have Docker available on your host. Just run the following command. Set extensions to your preference.

docker run --pull always --rm -it -v /etc/blackhole:/etc/blackhole \
ghcr.io/boss-net/blackhole:latest updateBlackholeFile.py --auto \
--replace --extensions gambling porn

If you want to add custom blackhole or a whitelist, create either or both files as per the instructions and add the following arguments before ghcr.io/boss-net/blackhole:latest depending on which you wish to use.

-v "path/to/myblackhole:/blackhole/myblackhole" \
-v "path/to/whitelist:/blackhole/whitelist" \

You can rerun this exact command later to update based on the latest available blackhole (for example, add it to a weekly cron job).

Option 2: Generate your own container image

We provide the Dockerfile used by the previous step, which you can use to create a container image with everything you need. The container will contain Python 3 and all its dependency requirements, and a copy of the latest version of this repository.

Build the Docker container from the root of this repo like this:

docker build --no-cache . -t bossnet-blackhole

Then run your command as such:

docker run --rm -it bossnet-blackhole updateBlackholeFile.py

This will create the blackhole file, and remove it with the container when done, so not very useful. You can use the example in option #1 to add volumes so files on your host are replaced.

Option 3: Generate it in your own environment

To generate your own amalgamated blackhole files you will need Python 3.6 or later.

First, install the dependencies with:

pip3 install --user -r requirements.txt

Note we recommend the --user flag which installs the required dependencies at the user level. More information about it can be found on pip documentation.

Option 4: Generate it in Google Colab

Spin up a free remote Google Colab environment.

Common steps regardless of your development environment

To run unit tests, in the top-level directory, run:

python3 testUpdateBlackholeFile.py

The updateBlackholeFile.py script will generate a unified blackhole file based on the sources in the local data/ subfolder. The script will prompt you whether it should fetch updated versions (from locations defined by the update.json text file in each source's folder). Otherwise, it will use the blackhole file that's already there.

python3 updateBlackholeFile.py [--auto] [--replace] [--ip nnn.nnn.nnn.nnn] [--extensions ext1 ext2 ext3]

Command line options

--help, or -h: display help.

--auto, or -a: run the script without prompting. When --auto is invoked,

Blackhole data sources, including extensions, are updated.
No extensions are included by default. Use the --extensions or -e flag to include any you want.
Your active blackhole file is not replaced unless you include the --replace flag.

--backup, or -b: Make a backup of existing blackhole file(s) as you generate over them.

--extensions <ext1> <ext2> <ext3>, or -e <ext1> <ext2> <ext3>: the names of subfolders below the extensions folder containing additional category-specific blackhole files to include in the amalgamation. Example: --extensions porn or -e social porn.

--flush-dns-cache, or -f: skip the prompt for flushing the DNS cache. Only active when --replace is also active.

--ip nnn.nnn.nnn.nnn, or -i nnn.nnn.nnn.nnn: the IP address to use as the target. Default is 0.0.0.0.

--keepdomaincomments, or -k: true (default) or false, keep the comments that appear on the same line as domains. The default is true.

--noupdate, or -n: skip fetching updates from blackhole data sources.

--output <subfolder>, or -o <subfolder>: place the generated source file in a subfolder. If the subfolder does not exist, it will be created.

--replace, or -r: trigger replacing your active blackhole

--skipstaticblackhole, or -s: false (default) or true, omit the standard section at the top, containing lines like 127.0.0.1 localhost. This is useful for configuring proximate DNS services on the local network.

--nogendata, or -g: false (default) or true, skip the generation of the readmeData.json file used for generating readme.md files. This is useful if you are generating host files with additional whitelists or blacklists and want to keep your local checkout of this repo unmodified.

--nounifiedblackhole: false (default) or true, do not include the unified blackhole file in the final blackhole file. Usually used together with --extensions.

--compress, or -c: false (default) or true, Compress the blackhole file ignoring non-necessary lines (empty lines and comments) and putting multiple domains in each line. Reducing the number of lines of the blackhole file improves the performances under Windows (with DNS Client service enabled).

--minimise, or -m: false (default) or true, like --compress, but puts each domain on a separate line. This is necessary because many implementations of URL blockers that rely on blackhole files do not conform to the standard which allows multiple blackhole on a single line.

--blacklist <blacklistfile>, or -x <blacklistfile>: Append the given blacklist file in blackhole format to the generated blackhole file.

--whitelist <whitelistfile>, or -w <whitelistfile>: Use the given whitelist file to remove blackhole from the generated blackhole file.

How do I control which sources are unified?

Add one or more additional sources, each in a subfolder of the data/ folder, and specify the url key in its update.json file.

Add one or more optional extensions, which originate from subfolders of the extensions/ folder. Again the url in update.json controls where this extension finds its updates.

Create an optional blacklist file. The contents of this file (containing a listing of additional domains in blackhole file format) are appended to the unified blackhole file during the update process. A sample blacklist is included, and may be modified as you need.

NOTE: The blacklist is not tracked by git, so any changes you make won't be overridden when you git pull this repo from origin in the future.

How do I include my own custom domain mappings?

If you have custom blackhole records, place them in file myblackhole. The contents of this file are prepended to the unified blackhole file during the update process.

The myblackhole file is not tracked by git, so any changes you make won't be overridden when you git pull this repo from origin in the future.

How do I prevent domains from being included?

The domains you list in the whitelist file are excluded from the final blackhole file.

The whitelist uses partial matching. Therefore if you whitelist google-analytics.com, that domain and all its subdomains won't be merged into the final blackhole file.

The whitelist is not tracked by git, so any changes you make won't be overridden when you git pull this repo from origin in the future.

How can I contribute blackhole records?

If you discover sketchy domains you feel should be included here, here are some ways to contribute them.

Option 1: contact one of our blackhole sources

The best way to get new domains included is to submit an issue to any of the data providers whose home pages are listed here. This is best because once you submit new domains, they will be curated and updated by the dedicated folks who maintain these sources.

Option 2: Fork this repository, add your domains to KhulnaSoft's personal data file, and submit a pull request

Fork this blackhole this repo and add your links to https://github.com/boss-net/blackhole/blob/master/data/boss-net/blackhole.

Then, submit a pull request.

WARNING: this is less desirable than Option 1 because the ongoing curation falls on us. So this creates more work for us.

Option 3: create your own blackhole list as a repo on GitHub

If you're able to curate your own collection of sketchy domains, then curate your own blackhole list. Then signal the existence of your repo as a new issue and we may include your new repo into the collection of sources we pull whenever we create new versions.

What is a blackhole file?

A blackhole file, named blackhole (with no file extension), is a plain-text file used by all operating systems to map hostnames to IP addresses.

In most operating systems, the blackhole file is preferential to DNS. Therefore if a domain name is resolved by the blackhole file, the request never leaves your computer.

Having a smart blackhole file goes a long way towards blocking malware, adware, and other irritants.

For example, to nullify requests to some doubleclick.net servers, adding these lines to your blackhole file will do it:

# block doubleClick's servers
0.0.0.0 ad.ae.doubleclick.net
0.0.0.0 ad.ar.doubleclick.net
0.0.0.0 ad.at.doubleclick.net
0.0.0.0 ad.au.doubleclick.net
0.0.0.0 ad.be.doubleclick.net
# etc...

We recommend using `0.0.0.0` instead of `127.0.0.1`

Traditionally most host files use 127.0.0.1, the loopback address, to establish an IP connection to the local machine.

We prefer to use 0.0.0.0, which is defined as a non-routable meta-address used to designate an invalid, unknown, or non-applicable target.

Using 0.0.0.0 is empirically faster, possibly because there's no wait for a timeout resolution. It also does not interfere with a web server that may be running on the local PC.

Why not use `0` instead of `0.0.0.0`?

We tried that. Using 0 doesn't work universally.

Location of your blackhole file

To modify your current blackhole file, look for it in the following places and modify it with a text editor.

macOS (until 10.14.x macOS Mojave), iOS, Android, Linux: /etc/blackhole file.
macOS Catalina: /private/etc/blackhole file.
Windows: %SystemRoot%\system32\drivers\etc\blackhole file.

Gentoo

Gentoo users may find sb-blackhole in ::pf4public Gentoo overlay

NixOS

To install blackhole file on your machine add the following into your configuration.nix:

{
  networking.extraBlackhole = let
    blackholePath = https://raw.githubusercontent.com/boss-net/blackhole/master/blackhole;
    blackholeFile = builtins.fetchurl blackholePath;
  in builtins.readFile "${blackholeFile}";
}

NOTE: Change blackholePath if you need other versions of blackhole file.
NOTE: The call to fetchurl is impure. Use fetchFromGitHub with the exact commit if you want to always get the same result.

Nix Flake

NixOS installations which are managed through flakes can use the blackhole file like this:

{
  inputs.blackhole.url = github:boss-net/blackhole;
  outputs = { self, nixpkgs, blackhole }: {
    nixosConfigurations.my-hostname = {
      system = "<architecture>";
      modules = [
        blackhole.nixosModule {
          networking.bossnetBlackHole.enable = true;
        }
      ];
    };
  };
}

The blackhole extensions are also available with the following options:

{
  networking.bossnetBlackHole = {
    blockFakenews = true;
    blockGambling = true;
    blockPorn = true;
    blockSocial = true;
  };
}

Updating blackhole file on Windows

(NOTE: See also some third-party Blackhole managers, listed below.)

On Linux and macOS, run the Python script. On Windows more work is required due to compatibility issues so it's preferable to run the batch file as follows:

updateBlackholeWindows.bat

This file MUST be run in command prompt with administrator privileges in the repository directory. In addition to updating the blackhole file, it can also replace the existing blackhole file, and reload the DNS cache. It goes without saying that for this to work, you must be connected to the internet.

To open a command prompt as administrator in the repository's directory, do the following:

Windows XP: Start → Run → cmd
Windows Vista, 7: Start Button → type cmd → right-click Command Prompt → "Run as Administrator"
Windows 8: Start → Swipe Up → All Apps → Windows System → right-click Command Prompt → "Run as Administrator"
Windows 10: Start Button → type cmd → right-click Command Prompt → "Run as Administrator"

You can also refer to the "Third-Party Blackhole Managers" section for further recommended solutions from third parties.

Warning: Using this `blackhole` file in Windows may require disabling DNS Cache service

Windows has issues with larger blackhole files. Recent changes in security within Windows 10 denies access to changing services via other tools except registry hacks. Use the disable-dnscache-service-win.cmd file to make proper changes to the Windows registry. You will need to reboot your device once that's done. See the the comments within the cmd file for more details.

Reloading blackhole file

Your operating system will cache DNS lookups. You can either reboot or run the following commands to manually flush your DNS cache once the new blackhole file is in place.

The Google Chrome browser may require manually cleaning up its DNS Cache on chrome://net-internals/#dns page to thereafter see the changes in your blackhole file. See: https://superuser.com/questions/723703

Windows

Open a command prompt with administrator privileges and run this command:

ipconfig /flushdns

Linux

Open a Terminal and run with root privileges:

Debian/Ubuntu sudo service network-manager restart
Linux Mint sudo /etc/init.d/dns-clean start
Linux with systemd: sudo systemctl restart network.service
Fedora Linux: sudo systemctl restart NetworkManager.service
Arch Linux/Manjaro with Network Manager: sudo systemctl restart NetworkManager.service
Arch Linux/Manjaro with Wicd: sudo systemctl restart wicd.service
RHEL/Centos: sudo /etc/init.d/network restart
FreeBSD: sudo service nscd restart

To enable the nscd daemon initially, it is recommended that you run the following commands:
```
sudo sysrc nscd_enable="YES"
sudo service nscd start
```
Then modify the blackhole line in your /etc/nsswitch.conf file to the following:
```
blackhole: cache files dns
```
NixOS: The nscd.service is automatically restarted when the option networking.extraBlackhole was changed.
Others: Consult this Wikipedia article.

macOS

As described in this article, open a Terminal and run:

sudo dscacheutil -flushcache;sudo killall -HUP mDNSResponder

Release management

This repository uses release-it, an excellent CLI release tool for GitHub repos and npm packages, to automate creating releases. This is why the package.json and .release-it.json files are bundled.

Goals of this unified blackhole file

The goals of this repo are to:

automatically combine high-quality lists of blackhole,
provide situation-appropriate extensions,
de-dupe the resultant combined list,
and keep the resultant file reasonably sized.

A high-quality source is defined here as one that is actively curated. A blackhole source should be frequently updated by its maintainers with both additions and removals. The larger the blackhole file, the higher the level of curation is expected.

It is expected that this unified blackhole file will serve both desktop and mobile devices under a variety of operating systems.

Third-Party Blackhole Managers

Unified Blackhole AutoUpdate (for Windows): The Unified Blackhole AutoUpdate package is purpose-built for this unified blackhole project as well as in active development by community members. You can install and uninstall any blacklist and keep it automatically up to date, and can be placed in a shared network location and deployed across an organization via group policies. And since it is in active development by community members, your bug reports, feature requests, and other feedback are most welcome.
ViHoMa is a Visual Blackhole file Manager, written in Java, by Christian Martínez. Check it out!

Interesting Applications

Blackhole-BL is a simple tool to handle blackhole file black lists. It can remove comments, remove duplicates, compress to 9 domains per line, add IPv6 entries. In addition, it can also convert black lists to multiple other black list formats compatible with other software, such as dnsmasq, DualServer, RPZ, Privoxy, and Unbound, to name a few.
Host Minder is a simple GUI that allows you to easily update your /etc/blackhole file to one of four consolidated blackhole files from boss-net/blackhole. It is provided as a deb package and comes pre-installed on UbuntuCE.
Maza ad blocking is a bash script that automatically updates host file. You can also update a fresh copy. And each time it generates a dnsmasq-compatible configuration file. Fast installation, compatible with MacOS, Linux and BSD.
Hostile is a nifty command line utility to easily add or remove domains from your blackhole file. If our blackhole files are too aggressive for you, you can use hostile to remove domains, or you can use hostile in a bash script to automate a post process each time you download fresh versions of blackhole.
macOS Scripting for Configuration, Backup and Restore helps customizing, re-installing and using macOS. It also provides a script to install and update the blackhole file using this project on macOS. In combination with a launchd it updates the blackhole file every x days (default is 4). To install both, download the GitHub repo and run the install script from the directory one level up.
Pi-hole is a network-wide DHCP server and ad blocker that runs on Raspberry Pi. Pi-hole uses this repository as one of its sources.
Block ads and malware via local BIND9 DNS server (for Debian, Raspbian & Ubuntu): Set up a local DNS server with a /etc/bind/named.conf.blocked file, sourced from here.
Block ads, malware, and deploy parental controls via local DualServer DNS/DHCP server (for BSD, Windows & Linux): Set up a blacklist for everyone on your network using the power of the unified blackhole reformatted for DualServer. And if you're on Windows, this project also maintains an update script to make updating DualServer's blacklist even easier.
Blocking ads and malwares with unbound – Unbound is a validating, recursive, and caching DNS resolver.
dnsmasq conversion script This GitHub gist has a short shell script (bash, will work on any 'nix) and uses wget & awk present in most distros, to fetch a specified blackhole file and convert it to the format required by dnsmasq. Supports IPv4 and IPv6. Designed to be used as either a shell script, or can be dropped into /etc/cron.weekly (or wherever suits). The script is short and easily edited, also has a short document attached with notes on dnsmasq setup.
BlackBlackhole - Command Line Installer/Updater This is a cross-platform command line utility to help install/update blackhole files found at this repository.
dnscrypt-proxy provides a tool to build block lists from local and remote lists in common formats.
Control D offers a public anycast network hosted mirror of the Unified (Adware + Malware) blocklist:
- Legacy DNS: 76.76.2.35, 76.76.10.35, 2606:1a40::35, 2606:1a40:1::35
- DNS-over-HTTPS/TLS/DOQ: https://freedns.controld.com/x-bossnet, x-bossnet.freedns.controld.com

Contribute

Please read our Contributing Guide. Among other things, this explains how we organize files and folders in this repository.

We are always interested in discovering well-curated sources of blackhole. If you find one, please open an issue to draw our attention.

Before you create or respond to any issue, please read our code of conduct.

Logo by @Tobaloidee Thank you!.

Name		Name	Last commit message	Last commit date
.github		.github
alternates		alternates
data		data
extensions		extensions
.dockerignore		.dockerignore
.editorconfig		.editorconfig
.gitattributes		.gitattributes
.gitignore		.gitignore
.npmrc		.npmrc
.release-it.json		.release-it.json
Dockerfile		Dockerfile
SECURITY.md		SECURITY.md
aggregator.png		aggregator.png
blackhole		blackhole
blackhole_file_size_history.png		blackhole_file_size_history.png
blacklist.example		blacklist.example
codebase_structure.md		codebase_structure.md
disable-dnscache-service-win.bat		disable-dnscache-service-win.bat
flake.nix		flake.nix
license.txt		license.txt
makeBlackhole.py		makeBlackhole.py
myblackhole.example		myblackhole.example
package.json		package.json
readme.md		readme.md
readmeData.json		readmeData.json
readme_template.md		readme_template.md
requirements.txt		requirements.txt
setup.cfg		setup.cfg
sourcestats.nb		sourcestats.nb
sourcestats.sh		sourcestats.sh
stats.nb		stats.nb
stats.out		stats.out
stats.sh		stats.sh
testUpdateBlackholeFile.py		testUpdateBlackholeFile.py
updateBlackholeFile.py		updateBlackholeFile.py
updateBlackholeWindows.bat		updateBlackholeWindows.bat
updateReadme.py		updateReadme.py
whitelist.example		whitelist.example

Uh oh!

Latest commit

History

Folders and files

Repository files navigation

Unified blackhole file with base extensions

List of all blackhole file variants

Sources of blackhole data unified in this variant

Extensions

Generate your own unified blackhole file

Option 1: Use our container image (Linux only)

Option 2: Generate your own container image

Option 3: Generate it in your own environment

Option 4: Generate it in Google Colab

Common steps regardless of your development environment

Command line options

How do I control which sources are unified?

How do I include my own custom domain mappings?

How do I prevent domains from being included?

How can I contribute blackhole records?

Option 1: contact one of our blackhole sources

Option 2: Fork this repository, add your domains to KhulnaSoft's personal data file, and submit a pull request

Option 3: create your own blackhole list as a repo on GitHub

What is a blackhole file?

We recommend using 0.0.0.0 instead of 127.0.0.1

Why not use 0 instead of 0.0.0.0?

Location of your blackhole file

Gentoo

NixOS

Nix Flake

Updating blackhole file on Windows

Warning: Using this blackhole file in Windows may require disabling DNS Cache service

Reloading blackhole file

Windows

Linux

macOS

Release management

Goals of this unified blackhole file

Third-Party Blackhole Managers

Interesting Applications

Contribute

About

Resources

Code of conduct

Security policy

Stars

Watchers

Forks

Releases

Sponsor this project

Packages

Used by

Contributors

Languages

We recommend using `0.0.0.0` instead of `127.0.0.1`

Why not use `0` instead of `0.0.0.0`?

Warning: Using this `blackhole` file in Windows may require disabling DNS Cache service