Verified Commit 01441d6f authored by Aral Balkan's avatar Aral Balkan
Browse files

Initial add of content and themes repositories; remove update scripts

parent 2bda8481
Pipeline #854 canceled with stages
2016-06-22 Major update to how we’re handling Google. Much more granular block rules. Fixes Google Maps embeds & Recaptcha.
2016-06-11 Blocking rules now more specific. Fixed Google images, eBay images issues. Better performance.
2016-06-08 Fixed Google News.
2016-06-07 Blocking flite.com (full-screen video doorslam)
2016-06-06 Fixed yimg tracker blocking Yahoo properties.
2016-06-05 Fixed Facebook image uploads.
2016-06-04 Fixed Twitter logins, now blocking Google AdSense on StartPage.com
2016-06-03 Initial release.
# Contributing Content to Better
**Thank you for your interest in contributing to Better.**
All content submitted to this repository is licensed under a Creative Commons Attribution-ShareAlike 4.0 International Public License.
You do *not* have to sign the [Better Contribution License Agreement](https://source.ind.ie/better/better/blob/master/CONTRIBUTING.md) to contribute content to this repository. (This is because Creative Commons Attribution-ShareAlike 4.0 International Public License is compatible with the App Store.)
Creative Commons Attribution-ShareAlike 4.0 International Public License
By exercising the Licensed Rights (defined below), You accept and agree to be bound by the terms and conditions of this Creative Commons Attribution-ShareAlike 4.0 International Public License ("Public License"). To the extent this Public License may be interpreted as a contract, You are granted the Licensed Rights in consideration of Your acceptance of these terms and conditions, and the Licensor grants You such rights in consideration of benefits the Licensor receives from making the Licensed Material available under these terms and conditions.
Section 1 – Definitions.
Adapted Material means material subject to Copyright and Similar Rights that is derived from or based upon the Licensed Material and in which the Licensed Material is translated, altered, arranged, transformed, or otherwise modified in a manner requiring permission under the Copyright and Similar Rights held by the Licensor. For purposes of this Public License, where the Licensed Material is a musical work, performance, or sound recording, Adapted Material is always produced where the Licensed Material is synched in timed relation with a moving image.
Adapter's License means the license You apply to Your Copyright and Similar Rights in Your contributions to Adapted Material in accordance with the terms and conditions of this Public License.
BY-SA Compatible License means a license listed at creativecommons.org/compatiblelicenses, approved by Creative Commons as essentially the equivalent of this Public License.
Copyright and Similar Rights means copyright and/or similar rights closely related to copyright including, without limitation, performance, broadcast, sound recording, and Sui Generis Database Rights, without regard to how the rights are labeled or categorized. For purposes of this Public License, the rights specified in Section 2(b)(1)-(2) are not Copyright and Similar Rights.
Effective Technological Measures means those measures that, in the absence of proper authority, may not be circumvented under laws fulfilling obligations under Article 11 of the WIPO Copyright Treaty adopted on December 20, 1996, and/or similar international agreements.
Exceptions and Limitations means fair use, fair dealing, and/or any other exception or limitation to Copyright and Similar Rights that applies to Your use of the Licensed Material.
License Elements means the license attributes listed in the name of a Creative Commons Public License. The License Elements of this Public License are Attribution and ShareAlike.
Licensed Material means the artistic or literary work, database, or other material to which the Licensor applied this Public License.
Licensed Rights means the rights granted to You subject to the terms and conditions of this Public License, which are limited to all Copyright and Similar Rights that apply to Your use of the Licensed Material and that the Licensor has authority to license.
Licensor means the individual(s) or entity(ies) granting rights under this Public License.
Share means to provide material to the public by any means or process that requires permission under the Licensed Rights, such as reproduction, public display, public performance, distribution, dissemination, communication, or importation, and to make material available to the public including in ways that members of the public may access the material from a place and at a time individually chosen by them.
Sui Generis Database Rights means rights other than copyright resulting from Directive 96/9/EC of the European Parliament and of the Council of 11 March 1996 on the legal protection of databases, as amended and/or succeeded, as well as other essentially equivalent rights anywhere in the world.
You means the individual or entity exercising the Licensed Rights under this Public License. Your has a corresponding meaning.
Section 2 – Scope.
License grant.
Subject to the terms and conditions of this Public License, the Licensor hereby grants You a worldwide, royalty-free, non-sublicensable, non-exclusive, irrevocable license to exercise the Licensed Rights in the Licensed Material to:
reproduce and Share the Licensed Material, in whole or in part; and
produce, reproduce, and Share Adapted Material.
Exceptions and Limitations. For the avoidance of doubt, where Exceptions and Limitations apply to Your use, this Public License does not apply, and You do not need to comply with its terms and conditions.
Term. The term of this Public License is specified in Section 6(a).
Media and formats; technical modifications allowed. The Licensor authorizes You to exercise the Licensed Rights in all media and formats whether now known or hereafter created, and to make technical modifications necessary to do so. The Licensor waives and/or agrees not to assert any right or authority to forbid You from making technical modifications necessary to exercise the Licensed Rights, including technical modifications necessary to circumvent Effective Technological Measures. For purposes of this Public License, simply making modifications authorized by this Section 2(a)(4) never produces Adapted Material.
Downstream recipients.
Offer from the Licensor – Licensed Material. Every recipient of the Licensed Material automatically receives an offer from the Licensor to exercise the Licensed Rights under the terms and conditions of this Public License.
Additional offer from the Licensor – Adapted Material. Every recipient of Adapted Material from You automatically receives an offer from the Licensor to exercise the Licensed Rights in the Adapted Material under the conditions of the Adapter’s License You apply.
No downstream restrictions. You may not offer or impose any additional or different terms or conditions on, or apply any Effective Technological Measures to, the Licensed Material if doing so restricts exercise of the Licensed Rights by any recipient of the Licensed Material.
No endorsement. Nothing in this Public License constitutes or may be construed as permission to assert or imply that You are, or that Your use of the Licensed Material is, connected with, or sponsored, endorsed, or granted official status by, the Licensor or others designated to receive attribution as provided in Section 3(a)(1)(A)(i).
Other rights.
Moral rights, such as the right of integrity, are not licensed under this Public License, nor are publicity, privacy, and/or other similar personality rights; however, to the extent possible, the Licensor waives and/or agrees not to assert any such rights held by the Licensor to the limited extent necessary to allow You to exercise the Licensed Rights, but not otherwise.
Patent and trademark rights are not licensed under this Public License.
To the extent possible, the Licensor waives any right to collect royalties from You for the exercise of the Licensed Rights, whether directly or through a collecting society under any voluntary or waivable statutory or compulsory licensing scheme. In all other cases the Licensor expressly reserves any right to collect such royalties.
Section 3 – License Conditions.
Your exercise of the Licensed Rights is expressly made subject to the following conditions.
Attribution.
If You Share the Licensed Material (including in modified form), You must:
retain the following if it is supplied by the Licensor with the Licensed Material:
identification of the creator(s) of the Licensed Material and any others designated to receive attribution, in any reasonable manner requested by the Licensor (including by pseudonym if designated);
a copyright notice;
a notice that refers to this Public License;
a notice that refers to the disclaimer of warranties;
a URI or hyperlink to the Licensed Material to the extent reasonably practicable;
indicate if You modified the Licensed Material and retain an indication of any previous modifications; and
indicate the Licensed Material is licensed under this Public License, and include the text of, or the URI or hyperlink to, this Public License.
You may satisfy the conditions in Section 3(a)(1) in any reasonable manner based on the medium, means, and context in which You Share the Licensed Material. For example, it may be reasonable to satisfy the conditions by providing a URI or hyperlink to a resource that includes the required information.
If requested by the Licensor, You must remove any of the information required by Section 3(a)(1)(A) to the extent reasonably practicable.
ShareAlike.
In addition to the conditions in Section 3(a), if You Share Adapted Material You produce, the following conditions also apply.
The Adapter’s License You apply must be a Creative Commons license with the same License Elements, this version or later, or a BY-SA Compatible License.
You must include the text of, or the URI or hyperlink to, the Adapter's License You apply. You may satisfy this condition in any reasonable manner based on the medium, means, and context in which You Share Adapted Material.
You may not offer or impose any additional or different terms or conditions on, or apply any Effective Technological Measures to, Adapted Material that restrict exercise of the rights granted under the Adapter's License You apply.
Section 4 – Sui Generis Database Rights.
Where the Licensed Rights include Sui Generis Database Rights that apply to Your use of the Licensed Material:
for the avoidance of doubt, Section 2(a)(1) grants You the right to extract, reuse, reproduce, and Share all or a substantial portion of the contents of the database;
if You include all or a substantial portion of the database contents in a database in which You have Sui Generis Database Rights, then the database in which You have Sui Generis Database Rights (but not its individual contents) is Adapted Material, including for purposes of Section 3(b); and
You must comply with the conditions in Section 3(a) if You Share all or a substantial portion of the contents of the database.
For the avoidance of doubt, this Section 4 supplements and does not replace Your obligations under this Public License where the Licensed Rights include other Copyright and Similar Rights.
Section 5 – Disclaimer of Warranties and Limitation of Liability.
Unless otherwise separately undertaken by the Licensor, to the extent possible, the Licensor offers the Licensed Material as-is and as-available, and makes no representations or warranties of any kind concerning the Licensed Material, whether express, implied, statutory, or other. This includes, without limitation, warranties of title, merchantability, fitness for a particular purpose, non-infringement, absence of latent or other defects, accuracy, or the presence or absence of errors, whether or not known or discoverable. Where disclaimers of warranties are not allowed in full or in part, this disclaimer may not apply to You.
To the extent possible, in no event will the Licensor be liable to You on any legal theory (including, without limitation, negligence) or otherwise for any direct, special, indirect, incidental, consequential, punitive, exemplary, or other losses, costs, expenses, or damages arising out of this Public License or use of the Licensed Material, even if the Licensor has been advised of the possibility of such losses, costs, expenses, or damages. Where a limitation of liability is not allowed in full or in part, this limitation may not apply to You.
The disclaimer of warranties and limitation of liability provided above shall be interpreted in a manner that, to the extent possible, most closely approximates an absolute disclaimer and waiver of all liability.
Section 6 – Term and Termination.
This Public License applies for the term of the Copyright and Similar Rights licensed here. However, if You fail to comply with this Public License, then Your rights under this Public License terminate automatically.
Where Your right to use the Licensed Material has terminated under Section 6(a), it reinstates:
automatically as of the date the violation is cured, provided it is cured within 30 days of Your discovery of the violation; or
upon express reinstatement by the Licensor.
For the avoidance of doubt, this Section 6(b) does not affect any right the Licensor may have to seek remedies for Your violations of this Public License.
For the avoidance of doubt, the Licensor may also offer the Licensed Material under separate terms or conditions or stop distributing the Licensed Material at any time; however, doing so will not terminate this Public License.
Sections 1, 5, 6, 7, and 8 survive termination of this Public License.
Section 7 – Other Terms and Conditions.
The Licensor shall not be bound by any additional or different terms or conditions communicated by You unless expressly agreed.
Any arrangements, understandings, or agreements regarding the Licensed Material not stated herein are separate from and independent of the terms and conditions of this Public License.
Section 8 – Interpretation.
For the avoidance of doubt, this Public License does not, and shall not be interpreted to, reduce, limit, restrict, or impose conditions on any use of the Licensed Material that could lawfully be made without permission under this Public License.
To the extent possible, if any provision of this Public License is deemed unenforceable, it shall be automatically reformed to the minimum extent necessary to make it enforceable. If the provision cannot be reformed, it shall be severed from this Public License without affecting the enforceability of the remaining terms and conditions.
No term or condition of this Public License will be waived and no failure to comply consented to unless expressly agreed to by the Licensor.
Nothing in this Public License constitutes or may be interpreted as a limitation upon, or waiver of, any privileges and immunities that apply to the Licensor or You, including from the legal processes of any jurisdiction or authority.
\ No newline at end of file
# Better activation test rule.
## Important!
**Do not edit or remove!** This rule is essential for the proper functioning of the iOS app.
This rule blocks the `test.js` script. The Better iOS app loads the `index.html` file which tries to load this script. If the blocker is inactive, the script will load and the browser will be redirected to a URL that signals that the blocker is inactive. If blocked, execution falls through to some inline JavaScript that forwards the browser to a URL that signals that the blocker is active.
Even though this rule is hardcoded into the blockerList.json that’s in the Blocker content blocker extension, that’s only used until the actual data is downloaded via Git. After that, the blockerList.json file from the data-for-apps repository is used. This is the file that adds the rule to that (main) blocker list file.
```mson
- trigger:
- url-filter: localhost:8000/test.js
- load-type: first-party
- action:
- type: block
```
#!/bin/sh
#
# Deploys content changes to production.
# (Auto-generates tag label and pushes tags to live.)
#
# (Make sure you have commited your changes and saved them to production using the ./save script first.)
#
set -e
# Aborts the script and decrements the deployment counter
abortGracefully(){
currentDayDeploymentCount=$(( currentDayDeploymentCount - 1 ))
echo "$currentDayDeploymentCount" > "$counterFilePath"
exit 1
}
dateStamp=$(date "+%Y%m%d")
counterFolderPath="/tmp/better-content"
counterFilePath="${counterFolderPath}/$dateStamp"
#
# Set up the counter, if necessary.
#
mkdir -p "$counterFolderPath"
if ! [ -f "$counterFilePath" ]; then
# Counter does not exist for this day, create it.
echo 0 > "$counterFilePath"
fi
#
# Increment the counter.
#
currentDayDeploymentCount=$(( $(cat "$counterFilePath") + 1 ))
# Persist the counter.
echo "$currentDayDeploymentCount" > "$counterFilePath"
# Format the counter (padded to three spaces).
printf -v formattedCurrentDayDeploymentCount "%03d" $currentDayDeploymentCount
# Create the git tag label
gitTagLabel="${dateStamp}-${formattedCurrentDayDeploymentCount}"
echo "\nTagging: ${gitTagLabel}"
#
# Create a default tag message based on the last commit message.
#
tempTagMessageDirectory=/tmp/better-content/
tempTagMessageFilePath="${tempTagMessageDirectory}/tag-commit-message.txt"
mkdir -p $tempTagMessageDirectory
# Not sure if there’s an easier way to get this.
gitDefaultEditor=$(git config -l | grep core.editor | perl -pe 's|^core.editor=||')
lastCommitMessage=$(git log -1 --oneline | perl -pe 's|^.*? ||')
echo "$lastCommitMessage" > "$tempTagMessageFilePath"
# Write out some instructions (so that it also resembles what people normally
# see when git opens up the editor. With an additional line of instructions on
# how to abort the tag.)
echo "\n#\n# Write a message for tag:\n#\t${gitTagLabel}\n# Lines starting with '#' will be ignored.\n#\n# If you’re happy with the default message above, just close this document.\n#\n# To abort deployment, delete all content and save the empty document." >> "$tempTagMessageFilePath"
# Create and open a temporary file to hold the message to pass to git tag later.
$gitDefaultEditor "$tempTagMessageFilePath"
emptyStringMD5=$(md5 -q -s '')
tagMessageMD5=$(md5 -q "$tempTagMessageFilePath")
# Check for empty tag message (this signals that the deployment should be aborted).
if [ "$emptyStringMD5" = "$tagMessageMD5" ]; then
# Person deleted the tag message, abort the tag.
echo "\nAborting deployment due to empty tag message.\n"
abortGracefully
else
echo ''
cat "$tempTagMessageFilePath"
fi
# Create the git tag
git tag -s "$gitTagLabel" --file="$tempTagMessageFilePath"
echo "Deploying…\n"
# Push the git tags
git push live --tags
echo "\nDeployment complete.\n\nSee https://www.better.fyi and https://data.better.fyi for changes.\n"
# All third-party scripts
This rule blocks all third-party scripts. This will break things. Enable only for testing.
<!--
```mson
.*
```
-->
# Better Builder
To get started, please run the `./dev` script.
<h1 class="legal">Legal and Privacy</h1>
**We exist to protect your privacy.**
<h2 id="version">Version 2018.1, as published on 13th September, 2018.</h2>
Our privacy policy is simple (and will stay that way). If we make any changes, we will let you know through the [Ind.ie news](https://ind.ie/news) and social media feeds. As this web site is open source (and all of its content released under a Creative Commons Attribution License), you can also [see a history of all changes](https://source.ind.ie/better/content/commits/master/legal/index.md).
## Privacy Policy
We do not track you. We do not collect or share your personal information.
Privacy is a fundamental human right that your other rights depend on. We exist to design beautiful, everyday things that protect your human rights. So, in a nutshell, our privacy policy is that we exist to protect your privacy.
### Analytics
The only analytics data we have are the server logs saved by default on nginx. We do not use this data, or share it with anybody else.
## Copyright
Better is copyright © 2015-2018 Article 12.
## Trademarks
Ind.ie, the Ind.ie balloon logo, and Better are trademarks of Article 12.
## Licenses
Better is free and open source (FOSS).
All content on the Better web site, excluding our trademarks and unless otherwise stated, is licensed under [Creative Commons Attribution-ShareAlike 4.0 International](https://creativecommons.org/licenses/by-sa/4.0/).
[The Better source code](https://source.ind.ie/better), unless otherwise stated, is available under [GNU Affero General Public License (AGPL) Version 3 or Later](http://www.gnu.org/licenses/agpl-3.0.html).
\ No newline at end of file
# Better
Better protects you from unethical web sites. It makes your web experience safer, lighter, and faster.
Better enforces the [Ethical Design Manifesto](https://ind.ie/ethical-design). It helps the Web respect human rights, effort, and experience.
Better is curated by Ind.ie, a tiny two-person-and-one-husky social enterprise striving for social justice in the digital age. Better is free, open, and transparent.
## Content
This repository contains the Better content: Better’s database of information on trackers and other malware as well as the web sites that host them.
This content is in Blockdown format. Blockdown is an extension of Markdown with special vocabulary to describe web malware. Blockdown can also contain WebKit content blocking rules. The Blockdown pages in Better’s content repository both describe web malware and contain the rules to block them.
This content is processed by [Better Builder](https://source.ind.ie/better/builder) to generate the [Better web site](https://better.fyi) as well as the data for the [Better iOS App](https://source.ind.ie/better/app), including a WebKit `blockerList.json` file.
A seminal advantage of Better is that its database is human-readable, open, and extensible via pull requests. (The database is curated by Ind.ie using the Ethical Design Manifesto as the criteria for blocking rules.)
Contributing to the content is as easy as creating an account on [source.ind.ie](https://source.ind.ie) and editing a content page in your browser.
## I’m not a developer, I just want to experience a Better web.
[Get Better from the App Store.](https://itunes.apple.com/us/app/better-by-ind.ie/id1080964978?mt=8)
## How can I support Better?
Buying [Better on the App Store](https://itunes.apple.com/us/app/better-by-ind.ie/id1080964978?mt=8) is one way to support us. If you want to help with the ongoing costs of developing and maintaining Better, you can [donate to Ind.ie](https://ind.ie/fund/) or, even better, [become a patron](https://ind.ie/fund/) by setting up a recurring donation.
## I’m a developer, let me in!
The easiest way to get started is to follow the instructions in the readme for the [Better iOS app](https://source.ind.ie/better/app) repository.
## Testing locally.
[Better Builder](https://source.ind.ie/better/builder) will automatically pick up your changes as you save and rebuild your local data.
To persist your changes locally, commit them in Git and push to origin:
```bash
git commit -am "My awesome content update"
git push origin master
```
Note that these changes will be destroyed if you run the Better Builder installer (or the Better iOS installer, which runs the Better Builder installer as part of its own installation process). To not lose any work, save your changes regularly by pushing to production, as explained below.
## Saving your changes by pushing to production
You can push to production with:
```bash
./save
```
Or, manually run what the save script does, which is:
```bash
git push live master
```
## Deployment
Before you can deploy, you must [set up a GPG key and configure Git to use it](https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work). This is used to sign your tags.
Then, if you have commit rights to the content repository, just run the deployment script:
```bash
./deploy
```
This will create a tag (you will have to enter a tag mesage when prompted, describing the release) and push it to production. Please make sure that you have already committed your changes and pushed them to production either via `git push live master` or by running the `./save` script, which does the same thing.
# Guide to Blockdown
Better content is authored in Blockdown.
Blockdown is Markdown with an extended high-level vocabulary for describing web malware for the Better knowledge base.
## Sites
Site pages have the following sections:
### Ethical design violations
```markdown
## Ethical design violations
```
This is a list of ethical design violations that gets converted to a collection of badges on the rendered site pages. The Trackers part of the list, detailed below, is updated automatically by [Better Inspector](https://source.ind.ie/better/inspector)
#### Trackers
The first badge is always the trackers badge. In Blockdown it is represented by a list item introduced by the word `Trackers`:
```markdown
* (Trackers)
* Automatically
* Generated
* List
* of
* Trackers
```
This gets automatically translated by [Better Builder](https://source.ind.ie/better/builder) to a badge similar to the one below:
![Screenshot of the trackers badge](images/readme/better/trackers-badge-example.png)
Tapping on the badge displays a popover with links to the actual trackers.
![Screenshot of the trackers popover](images/readme/better/trackers-popover-example.png)
The other badges are manually added if they apply to the site in question:
#### Aggressive
```markdown
* (Aggressive)
```
Attempts to block content blockers.
![Screenshot of the Aggressive Badge](images/readme/better/aggressive-badge-example.png)
#### Doorslam
```markdown
* (Doorslam)
```
Interrupts and blocks using modal dialogs.
![Screenshot of the Doorslam Badge](images/readme/better/doorslam-badge-example.png)
#### Clickbait
```markdown
* (Clickbait)
```
Uses exploitative, addictive content syndication network(s).
![Screenshot of the Clickbait Badge](images/readme/better/clickbait-badge-example.png)
#### Fingerprint
```markdown
* (Fingerprint)
```
Uses hidden Canvas fingerprinting.
![Screenshot of the Fingerprint Badge](images/readme/better/fingerprint-badge-example.png)
#### Web Bug
```markdown
* (Web bug)
```
Uses invisible tracking pixels.
![Screenshot of the Web Bugs Badge](images/readme/better/web-bugs-badge-example.png)
We might create new badges as and when we find new types of web malware and unethical practices to document and warn people about.
## After Better section
```markdown
## After Better
```
The After Better section provides statistics about the before (without the Better content blocker active) and after (with the Better content blocker active) performance of a site.
It is automatically generated by [Better Inspector](https://source.ind.ie/better/inspector)
![Screenshot of the After Better Section](images/readme/better/after-better.png)
## Block Rules section
This is the section where we enter the actual WebKit content blocking rules. Each rule is written in a strict subset of MSON (Markdown JSON) and has a brief explanation detailing what the rule does and why.
The blocking rules in this section serve the following purposes, in line with the [Ethical Design Manifesto](https://ind.ie/ethical-design)
* Remove any first-party trackers (respect human rights)
* Improve the usability of the site by removing first-party impediments like doorslams (respect human effort)
* Improve the experience of the site (respect human effort) – we should especially aim to create a better experience after trackers have been removed (like removing empty spaces left over, etc.)
Please note that this is not the place to put blocking rules for trackers. Each tracker encountered should be entered into the [Trackers](#trackers) section and you have its own page in the `/trackers` section of the content.
### Blockdown syntax
Here is an example of a site-specific blocking rule in Blockdown format:
```markdown
```mson
- trigger:
- url-filter: cdn.cultofmac.com/wp-content/plugins/com2014-ads/static/js/frontend-functionality.js
- action:
- type: block
``` 
```
The Blockdown parser in Better supports all of the [WebKit content blocking rules](https://webkit.org/blog/3476/content-blockers-first-look/). Instead of JSON, however, we enter blocking rules in MSON. All Blockdown rules are combined by Better Builder into a single `blockerList.json` file.
Blockdown differs from plain WebKit content blocker rules in several ways to make authoring easier and to aid in readability:
1. The default load type is ‘third-party’.
2. The default action type is ‘block’.
2. The default is for rules to be case sensitive.
So, if we take the following fully-specified rule:
```markdown
```mson
- trigger:
- url-filter: somedomain.ext
- load-type: third-party
- url-filter-is-case-sensitive: true
- action
- type: block
``` 
```
We can simplify it naïvely by removing the properties that have defaults:
```markdown
```mson
- trigger:
- url-filter: somedomain.ext
- action
``` 
```
Which leaves us with a valid rule but a sad-looking empty action section. In Blockdown neither the trigger nor action sections are required, so we can remove those also. This leaves us with:
```markdown
```mson
- url-filter: somedomain.ext
``` 
```
But surely, we can do better than that. So we handle this special case in Blockdown by not requiring the url-filter key either:
```markdown
```mson
somedomain.ext
``` 
```
Ah, better! ;)
All of the above Blockdown rules are equivalent and will compile into the following fully-formed and highly specific WebKit content blocking rule in JSON:
```json
{
"trigger": {
"load-type": [
"third-party"
],
"url-filter-is-case-sensitive": true,
"url-filter": "^[^:]+://+([^:/]+\\.)?somedomain\\.ext[:/]?"
},
"action": {
"type": "block"
}
}
```
## Automatic URL filter compilation
Blockdown automatically compiles simple `url-filter` properties to regular expressions with higher specificity as recommended in the [domain targeting recommendations by WebKit engineer Benjamin Poulain](https://webkit.org/blog/4062/targeting-domains-with-content-blockers/).
This means that you can author your entries in plain text, like this:
```markdown
- url-filter: some-domain.ext
```
And Blockdown will compile them into the following form in the blockerList.json:
```json
"url-filter": "^[^:]+://+([^:/]+\\.)?some-domain\\.ext[:/]?"
```
## Further reading on WebKit content blocking
* [Introduction to WebKit Content Blockers](https://webkit.org/blog/3476/content-blockers-first-look/)
* [Targeting Domains with Content Blockers](https://webkit.org/blog/4062/targeting-domains-with-content-blockers/)
* [Official Safari content-blocking rules documentation from Apple](https://developer.apple.com/library/mac/documentation/Extensions/Conceptual/ContentBlockingRules/Introduction/Introduction.html)
# Investigation process
Currently, you need to have commit rights to the Content repository to use the Better commandline commands. However, you can use Git directly to fork the repository and submit merge requests and you can [add and edit pages through the online GitLab interface](https://source.ind.ie/better/content) without commit rights.
## Find who owns and runs the tracker
1. **Start by editing the tracker**
```bash
better/edit drafts/trackers/somedoma.in
```
This will create an issue in GitLab (or update an existing issue, if one already exists) and create or checkout a branch for you. It will also open your working copy of the tracker page in your system editor and in the browser.
2. **Enter the tracker URL into your browser in a private window to see if it loads.**
Make sure you don’t have an VPNs or extensions blocking or making your browser behave differently from the norm. If you have any tracker blockers already enabled, it may make it harder to investigate!
3. **If it doesn’t load, or if you get a blank page, perform a whois.**
We are currently using http://whois.domaintools.com for these so we can link to is as a source when stating ownership information. However, you will sometimes get more information from a direct whois look-up on your machine. In Terminal: `whois somedoma.in`
4. **Some trackers use a domain proxy or a cloaking service** (e.g., Domains by Proxy) to further hide their origins. In this case:
* Open up the source of a site that the tracker originated on in the Web Developer console (Timeline view) of Safari (or in the web inspector of your browser of choice)
* Try to recreate the original call. This might give you more clues about its origin.
To find which sites a tracker is on, perform a search on the ~/better.fyi/drafts/trackers folder. For example, you can open up the folder in your text editor and do a global search for the tracker name.
You can also use [Better Inspector](https://source.ind.ie/better/inspector) to search for strings within requests. e.g., to find all URLs that contain *google.com*, run:
```bash
./inquiry --local --find=google.com
```
Other useful tools:
* [Mozilla Lightbeam](https://www.mozilla.org/en-US/lightbeam/)