Start collecting and analysing data
Start Mitmproxy. Now open the web viewer in the phone and look at mitmproxy – you should see the request coming in on the data interception environment. You can also summon instructions there as to how to set up the device, including information around if you want to use mitmproxy with devices running API level 24 or higher.
Open your web browser on the virtual phone and go to mitm.it and download the certificate. Once the certificate is downloaded it will appear in the download section of the files app.
Start your local terminal so you can install certificates into the system root store as outlined in the documentation on the GitHub and inside the DIE.
Run ADB devices to see which devices are available to connect to. There’s only one device available, which is the Genymotion device.
Then start ADB shell, which will allow you to access the local console of the Genymotion phone.
If you change the directory to SD card download and list the directory, you should see the mitmproxy certificate listed there. For the time being you can close mitmproxy as you don’t need it.
Copy and paste the open SSL command from below or from the local documentation into the local console in the VM running the DIE.
sudo openssl x509 -inform PEM -subject_hash_old -in /root/.mitmproxy/mitmproxy-ca-cert.pem | head -1
This will return the name you will later need to give the file in Android for it to recognize the certificate authority. This is how you’re going to become the local administrator on the virtual device.
When we copied the open SSL command into the console, we got the output C8750F0D. This is the name we’ll have to give the certificate later when we copy it into the system root store on the Genymotion device.
Go back to your local console, which should still be running ADB shell
First execute su to become super user on the device, i.e. the local administrator.
Remount the partition system folder. This allows us to modify system data, which is usually read only.
mount -o remount,rw /system
Copy the certificate from the downloads folder into the systems certificate folder. You can use this in the console: cp /sdcard/Download/mitmproxy-ca-cert.pem /system/etc/security/cacerts/<NamefromOpenSSLOutput>.0
When copying the file, you need to use the name you retrieved earlier running from running the open SSL command, for us that would be C8750F0D. And then the extension is .0 – making the full title C8750F0D.0
You now need to change the file permissions, so copy this into the console: chmod 644 /system/etc/security/cacerts/<NamefromOpenSSLOutput>.0
And again, type in the certificate’s name to replace <NamefromOpenSSLOutput>.
You also need to change who the file belongs to and make sure it belongs to root: chown root:root /system/etc/security/cacerts/NamefromOpenSSLOutput>.0
Now remount the file system read only: mount -o remount,ro /system
Reboot the device.
Exit the console and restart mitmproxy. You should see on the left that the device is now rebooting. Open mitmproxy in the web browser – you should be able to see a large amount of data that’s being collected. This is because as the device boots up it checks for internet access.
Look at the WiFi settings, it should now just say ‘connected’, unlike before where it said ‘connected, no internet’.
You can check that mitmproxy is installed correctly, by opening your browser and going to a website. You can go to the list of things being intercepted by mitmproxy and you’ll be able to see all the requests being made to the website.
Then, if click on the status bar, you should see that the certificate being presented is from mitmproxy, and if you look at the certificate information you should see that the assurer is mitmproxy.
Just for the sake of completeness, go to Privacy International’s website not on the phone and you should see your browser isn’t being intercepted. Again, look at the certificate information – this certificate is usually issued by DigiSure, Inc. If you look at the certificate it even includes all the information about PI.
You can now close your browser, and all the other applications you had open – you don’t need any at the moment.
Open the settings app and do a search for certificates and look at the trusted credentials section. Displays, which displays the trusted CAs and click on the system section. You can scroll down and see that the mitmproxy is a trusted system root.
Close mitmproxy and start a new session. Every time mitmproxy is restarted, the previous capture is saved to disk. If you head back to mitmproxy’s webpage. You should see the capture is now clear as a new capture has been started. We would recommend always starting a new capture with every different application you wish to test.
Start testing apps
Open the first app you want to test.
For example, we tried the UK’s met office’s app – in the gif below you can see that it’s making a number of connections to various web services, a lot of them appear to be Amazon Web Services. This is before we’ve even accepted their privacy policy.
If you click on ‘request’, you should be able to see what the original request was and what the response was.
For example, this is Google Firebase and we can see what token was used to authenticate with and what response was given for that Firebase installation.
In the bottom right-hand corner you can also find out information such as the SDK conversion of this version of Firebase Analytics and you can see things like remote requests. You can also see downloaded imagery such as that displayed in the top portion of the Met Office app – in this case the logo.
As you click things in the app, such as accepting the privacy policy, you should see more connections being made – some will be content, others metrics, advertising, or other third party services.
Below is an example of a fairly standard message sent by Facebook’s SDK to graph.facebook. You can see there’s quite a lot of information you can glean from this message including the event that’s being sent to Facebook along with information about the device, the application, the advertising ID, all of which can be cross referenced by Facebook to create or add to a profile.
The DIE can tell you what’s being sent from your device and where to, but it cannot tell you want the company is doing with that information, what processing they are doing, or whether they are sending that data on to third parties. Above you can see what the app is sending to Facebook, but it can’t tell you what Facebook is doing with that information.
When you’re done with the app – you can close your mitmproxy session. All captures are stored locally in privacy’s home folder, so the most recent captures can all be reloaded. If you want to analyse another app – start a new mitmproxy session.
If you want to look at the data you’ve previously gathered you should be able to find it as below.
A note on Certificate Pinning
One of the new features of the data interception environment is the inclusion of APK MITM, which allows the removal of certificate pins. Some large apps, like Facebook, banking apps, Twitter and some of the other apps that have large numbers of users, use a technique called certificate pinning, which means that they expect certain certificates to be presented by the remote side by making a connection. And if the, the certificate differs, then they will not make the secured connection.
What APK MITM allows is the removal of those pins so you can continue to do analysis in the data interception environment.
Further Information
PI has made this tool available to assist individuals, researchers and organisations in developing a technical understanding of how their own data is being captured, processed and transferred by applications to 3rd parties. It should be used to analyse data on a device, virtual copy of a device, or an app, which the user owns or is legally authorised to use.
The Data Interception Environment software is free to use in accordance with terms of the GNU General Public License. However, Privacy International’s brand, logo and name are copyrighted and protected from unauthorised use. PI cannot endorse any modifications that users or any other third parties make to PI’s Data Interception Environment and attribute to PI.
PI is a charity registered in England and Wales. We have developed our Data Interception Environment tool in accordance with the laws, regulations and standards that apply to us in England and Wales. Any individual, researcher or organisation who makes use of of PI’s Data Interception Environemnt is responsible for ensuring that they are acting lawfully and in accordance with the laws, regulations and standards which apply to them in the jurisdiction they are operating in or to which they are subject.
Enjoy the Data Interception Environment. If you’ve found the DIE useful, you can show your support by donating to make sure PI can continue to create things you find useful
