Charles

Charles is a powerful corss-platform proxy tool, it’s really helpful when debugging HTTPS network request, so that you can locate the problem in a flash ~

Introduce

Charles provides many powerful features so that it's really helpful in many scenarios

  1. View all the network traffic in detail
  2. Edit newtork request and response with breakpoint
  3. Request mapping , love this feature to death ~
  4. Bandwidth Throttling
  5. Network White & Black list
  6. The other key feature

Install

As a nice tutorial, i will only suggest you to buy a genuine license from the the Official Charles Website

If you are in China(天朝), feel free to get a copy from Lizhi.io to get a discount ~

Even if you don't want to pay for this awesome App for now, you can still trial the App for free forever !!!

The limitation of the trial version is :

  1. 30s waiting before launch app
  2. self-destruction every 30min

Install Charles is as simple as drag then drop, no more explanation here.

Launch Charles, and it will ask for permission to automatically configure your network settings.

Setup

1. HTTP Proxy

After launch the Charles App, the macOS Proxy is checked by default

Then open the browser and the http traffic will shown like this :

In Structure Mode : the traffic is organized by host

In Sequence Mode : the traffic will list one by one

Now feel free to navigate between the different section of the Charles ~

Try view the traffic of http://httpbin.org/image

2. Config HTTPS

Most of the morder App or website using HTTPS connection now, It necessary to support HTTPS traffic proxy, Charles SSL proxy feature

  1. Before enable HTTPS proxy :

    We even can't view the request url, all the request response detail is a messy ~

  1. After config HTTPS proxy :

    All the data is pure and clean, 666

  1. Simple install the root certificate, the keychain will open automatically, we need to trust this root certificate in order to config macOS proxy
  2. Enable SSL proxy for specific host

Now we can view all the HTTPS data ~

3. For iOS or Android Device

For iOS or Android Device, another certificate is necessary ~

As the prompt says, using Safari to view chls.pro/ssl then install the profile

For iOS 11 or above, an extra step is trust the root certificate in the

Settings => General => About Phone => Cetficate Trust Setting
设置 => 通用 => 关于本机 => 证书信任设置


Now certificate is installed correctly ~

Then we need to connnect Charles and out Device in the same LAN

  1. Connect to the same Wifi signal with your Mac

  1. Open the advanced confuration of the wifi connection

  1. Choose config proxy

Choose Manual config, then input the your Mac ip address and Charles proxy port (the default port should be 8888)

You can get the IP address from here or Mac System preference - Network

Basic Usage

1. Filter traffic

Using keywords to filter the traffic you need

2. Throttle

Enable the Throttle in the Proxy -> Throttle Settings

3. Repeat

Repeat request once or loop repeat concurrently

4. White Black list

Only block or accept request for black or white list

Advanced Usage

1. Map Remote

Right-click on a request, enable Mapping for specific request

This setting map all the requests match https://slack.com/clog/tracks/* to https://mywebsite.com/clog/tracks/*

It's extremely useful when switch hosts between Production QA Local Development Env

More settings available here ~

2. Map Local

Also, right-click on a request to enable Map local for specific request

Now all the request matching https://slack.com/clog/track/* will be intercepted and the response will be the content of Users/Shawn/Downloads/fake.json

It's extremely useful to create a mock server or modify the response continuously ~

3.BreakPoint

Right-click on a request or open the detail settings from Proxy -> Breakpoint Settings

Now, we will intercept all the traffic matching Get https://jianshu.com/*

Then navigate to https://jianshu.com , the request will be intercepted by Charles

we can edit request : such as modify request param headers cookies or abort request Then click Execute

After Charles got server response, Charles will break response

We can inspect the response detai and edit response here, the response from jianshu.com should be HTML, then click Execute the network request finished as usual ~

It's really an awesome feature ~

Extra

1. SSL Pinning

Trust any unknown Certificate is really dangerous, since it will expose all the HTTPS secure data to the others

So any solutions to protect us from MITM ?

:tada [SSL Pinning](https://en.wikipedia.org/wiki/HTTP_Public_Key_Pinning) ~

The way it work is pack the real certificate in the App, then verify the certificate on every connection ~

The facebook iOS App is well protected by SSL Pinning , when i try to debug the login request of the Facebook App, it fails ~

2. How https proxy works ?

man-in-the-middle

中间人攻击

In brief

For Normal Request

Server <----> Browser
         ^
       HTTPS

After Charles MITM attack

Server <---> Local Proxy <---> Browser
         ^                 ^
       HTTPS(1)          HTTPS(2)
  1. server on behalf of all the web service like google.com
  2. browser on behalf of our browser or the app on our phone
  3. Local Proxy is the proxy like Charles Wireshark whistle

The main routine :

  1. Browser send request
  2. Charles receive request and send it to real server
  3. Charles receive server response then foward the response to the Browser

It look really easy and simple, But in fact, HTTPS = HTTP + SSL/TLS, It's a secure protocol designed to protect our data from sniffer

So We need extra configuration for Charles to delegate all HTTPS secure traffic

FAQ

  1. Failed to install certificate from chls.pro/ssl

Somehow, if you fail to install the root certificate in your iOS device, try Save Charles root certificate on you Mac then email the cert to your phone and open the email attached cert then install

Ref