An article teaches you use Python to draw Android CPU and memory growth curves

Let's talk! ! ! !

When doing performance monitoring, it will be more intuitive if you can display the monitored CPU and memory growth changes in a graph. It took a while to implement it in Python. Let's see how to use Python to draw the Android CPU and memory change curve. PNG image of the growth curve chart.

1. result of implementation¶

At the beginning, I wanted to export the collected CPU and memory data to Excel to generate a growth curve chart. I did a survey, and there is no better way to achieve it. Later, I saw that it is easy to use Python to draw charts, and the learning cost of Python is low. Those who have done development such as grammar will know how to use it after a little look, and it is easy to get started.

The specific implementation effect is as follows. The data collected by CPU and memory are independent processes. The memory is divided into three pieces of data, the total application memory, Native memory and Dalvik memory. If there is a memory leak, either in Native or Dalvik, from the graph growth curve It's easy to see from above.

2. Detailed explanation of specific logic implementation¶

2.1, Python implementation of CPU chart code¶

show as below: 

# -*- coding: utf-8 -*-

import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
import json
import sys
import time
import traceback

def startDump():
  try:
    cpuData = json.loads(sys.argv[1])
    imagePath = sys.argv[2]
    cpuRateArray = []
    timeArray = []
    for cpuItem in cpuData:
      cpuRateArray.append(float(cpuItem["cpuRate"]))
      timeArray.append((float(float(cpuItem["time"]) - float(cpuData[0]["time"]))/1000))

    plt.title("Monitor Cpu Rate")
    plt.figure(figsize=(10, 8))
    plt.plot(timeArray, cpuRateArray, c='red', label='Process CPU')
    plt.ylabel("CPURate (%)", fontsize=12)
    plt.xlabel("TimeRange:" + formatTime(float(cpuData[0]["time"])) + ' - ' + formatTime(float(cpuData[len(cpuData) -1]["time"])), fontsize=10)
    plt.legend()
    plt.tight_layout()
    plt.savefig(imagePath)

  except Exception:
    print 'exeption occur:' + traceback.format_exc()

def formatTime(timeMillis):
  timeSeconds = float(timeMillis/1000)
  timelocal = time.localtime(timeSeconds)
  timeFormat = time.strftime("%Y-%m-%d %H:%M:%S", timelocal)
  return timeFormat

if __name__ == '__main__':
  startDump()

2.2, Python implementation of memory chart¶

code show as below:

# -*- coding: utf-8 -*-

import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
import json
import sys
import time
import traceback

def startDump():
  try:
    memoryData = json.loads(sys.argv[1])
    imagePath = sys.argv[2]
    totalPssArray = []
    nativePssArray = []
    dalvikPssArray = []
    timeArray = []
    for memoryItem in memoryData:
      totalPssArray.append(float(memoryItem["totalPss"])/1024)
      nativePssArray.append(float(memoryItem["nativePss"])/1024)
      dalvikPssArray.append(float(memoryItem["dalvikPss"])/1024)
      timeArray.append((float(float(memoryItem["time"]) - float(memoryData[0]["time"]))/1000))

    plt.title("Monitor Memory")
    plt.figure(figsize=(10, 8))
    plt.plot(timeArray, totalPssArray, c='red', label='Total Memory')
    plt.plot(timeArray, nativePssArray, c='yellow', label='Native Memory')
    plt.plot(timeArray, dalvikPssArray, c='blue', label='Dalvik Memory')
    plt.ylabel("Memory (MB)", fontsize=12)
    plt.xlabel("TimeRange:" + formatTime(float(memoryData[0]["time"])) + ' - ' + formatTime(float(memoryData[len(memoryData) -1]["time"])), fontsize=10)
    plt.legend()
    plt.tight_layout()
    plt.savefig(imagePath)

  except Exception:
    print 'exeption occur:' + traceback.format_exc()

def formatTime(timeMillis):
  timeSeconds = float(timeMillis/1000)
  timelocal = time.localtime(timeSeconds)
  timeFormat = time.strftime("%Y-%m-%d %H:%M:%S", timelocal)
  return timeFormat

if __name__ == '__main__':
  startDump()

3. Implementation instructions

There are two parameters passed by the script, one is the monitored JSON data string value sys.argv[1], and the other is the full path of the saved image file sys.argv[2]. The string value of the incoming JSON parameter needs to be decorated with single quotation marks, otherwise it will cause an abnormal parsing. The incoming JSON parameter cannot be a JSON object directly, and must be converted into a string. The example call command is as follows:

python dump_chart.py  '<JSONString>'  cpu_chart.png 

1. Sample CPU sample data, time is the system timestamp of the device, and the calculation of the CPU occupancy rate can be viewed in the back mask: CPU monitoring of Android performance monitoring.

[
 {
 "time": "1589435564442.279053",
 "cpuRate": "2.17"
 },
 {
 "time": "1589435565655.333008",
 "cpuRate": "3.26"
 },
 {
 "time": "1589435566954.137939",
 "cpuRate": "2.52"
 },
 ...
]

2. Sample memory sample data. The values of totalPss, nativePss and dalvikPss are all raw data intercepted from the application memory information output by dumpsys meminfo, corresponding to the Pss Total value of the "TOTAL", "Native Heap", and "Dalvik Heap" fields.

[
 {
 "time": "1589636256923.429932",
 "totalPss": 177804,
 "nativePss": 27922,
 "dalvikPss": 10212
 },
 {
 "time": "1589636258236.298096",
 "totalPss": 178021,
 "nativePss": 27850,
 "dalvikPss": 9990
 },
 {
 "time": "1589636259525.219971",
 "totalPss": 177899,
 "nativePss": 27742,
 "dalvikPss": 9990
 },
 ...
]

3.1. Problems encountered in the implementation process

1. Wrong use of load method

The json.load() method is used incorrectly and should be replaced with json.loads().

exeption occur:Traceback (most recent call last):
  File "*******", line 11, in startDump
    memoryData = json.load(sys.argv[1])
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 287, in load
    return loads(fp.read(),
AttributeError: 'str' object has no attribute 'read'

2. JSON string object entry problem

File "******", line 11, in startDump
    memoryData = json.loads(sys.argv[1])
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 382, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

For Python script calls, the JSON string object is used as the input parameter, and the incoming JSON string object needs to be processed with single quotes. For example, the example processing in JavaScript is as follows:

 '\'' + JSON.stringify(cpuRateJSON) + '\''

3.3 Python needs to display the type of the declared parameter¶

In Python, you need to specify the type of the parameter. After parsing the value obtained in the JSON object, Python does not determine the type according to the parameter. It needs to specify the object parameter type to be converted, such as converting the system timestamp to a float value type: float(memoryData[0][“time”]).

Traceback (most recent call last):
  File "*******", line 21, in startDump
    timeArray.append(timeStamp(memoryItem["time"]))
  File "*******", line 36, in timeStamp
    timeStamp = float(timeNum/1000)
TypeError: unsupported operand type(s) for /: 'unicode' and 'int'

4. Abnormalities caused by coding¶

SyntaxError: Non-ASCII character '\xe5' in file ******* on line 24, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details

If the following exception is reported after running, it means that there is a problem with the encoding, and the encoding type declaration is added at the beginning of the script:

#!usr/bin/python
# -*- coding: utf-8 -*-

5. Restrictions on the format of saved files¶

plt.savefig(image_path) can only save files in eps, pdf, pgf, png, ps, raw, rgba, svg, svgz formats. It does not support saving jpg images.

Traceback (most recent call last):
  File "/Users/chenwenguan/Documents/AmapAuto/Project/arc-resources/script/performanceMonitor/dump_cpu_chart_image.py", line 23, in startDump
    plt.savefig(image_path)
  File "/usr/local/lib/python2.7/site-packages/matplotlib/pyplot.py", line 695, in savefig
    res = fig.savefig(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/matplotlib/figure.py", line 2062, in savefig
    self.canvas.print_figure(fname, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/matplotlib/backend_bases.py", line 2173, in print_figure
    canvas = self._get_output_canvas(format)
  File "/usr/local/lib/python2.7/site-packages/matplotlib/backend_bases.py", line 2105, in _get_output_canvas
    .format(fmt, ", ".join(sorted(self.get_supported_filetypes()))))
ValueError: Format 'jpg' is not supported (supported formats: eps, pdf, pgf, png, ps, raw, rgba, svg, svgz)

6. python-tk dependency¶

Traceback (most recent call last):
  File "*******", line 2, in <module>
    import matplotlib.pyplot as plt
  File "/home/arc/.local/lib/python2.7/site-packages/matplotlib/pyplot.py", line 115, in <module>
    _backend_mod, new_figure_manager, draw_if_interactive, _show = pylab_setup()
  File "/home/arc/.local/lib/python2.7/site-packages/matplotlib/backends/__init__.py", line 63, in pylab_setup
    [backend_name], 0)
  File "/home/arc/.local/lib/python2.7/site-packages/matplotlib/backends/backend_tkagg.py", line 4, in <module>
    from . import tkagg  # Paint image to Tk photo blitter extension.
  File "/home/arc/.local/lib/python2.7/site-packages/matplotlib/backends/tkagg.py", line 5, in <module>
    from six.moves import tkinter as Tk
  File "/home/arc/.local/lib/python2.7/site-packages/six.py", line 203, in load_module
    mod = mod._resolve()
  File "/home/arc/.local/lib/python2.7/site-packages/six.py", line 115, in _resolve
    return _import_module(self.mod)
  File "/home/arc/.local/lib/python2.7/site-packages/six.py", line 82, in _import_module
    __import__(name)
  File "/usr/lib/python2.7/lib-tk/Tkinter.py", line 42, in <module>
    raise ImportError, str(msg) + ', please install the python-tk package'

Lack of python-tk dependency, execute the following command to install:

sudo apt-get install -y python-tk

7. Initial configuration of Agg canvas¶

Traceback (most recent call last):
  File "******", line 22, in startDump
    plt.title("ARC Monitor Memory")
  File "/home/arc/.local/lib/python2.7/site-packages/matplotlib/pyplot.py", line 1419, in title
    return gca().set_title(s, *args, **kwargs)
  File "/home/arc/.local/lib/python2.7/site-packages/matplotlib/pyplot.py", line 969, in gca
    return gcf().gca(**kwargs)
  File "/home/arc/.local/lib/python2.7/site-packages/matplotlib/pyplot.py", line 586, in gcf
    return figure()
  File "/home/arc/.local/lib/python2.7/site-packages/matplotlib/pyplot.py", line 533, in figure
    **kwargs)
  File "/home/arc/.local/lib/python2.7/site-packages/matplotlib/backend_bases.py", line 161, in new_figure_manager
    return cls.new_figure_manager_given_figure(num, fig)
  File "/home/arc/.local/lib/python2.7/site-packages/matplotlib/backends/_backend_tk.py", line 1046, in new_figure_manager_given_figure
    window = Tk.Tk(className="matplotlib")
  File "/usr/lib/python2.7/lib-tk/Tkinter.py", line 1828, in __init__
    self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)
TclError: no display name and no $DISPLAY environment variable

This problem does not occur when running on a Mac, but an exception is reported when running in an Ubuntu environment. The explanation on the official website is as follows:

When using Matplotlib versions older than 3.1, it is necessary to explicitly instantiate an Agg canvas

At the beginning of the script file, display the statement that Agg uses:

import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt

8. pyecharts version configuration problem¶

If you are not drawing in the native way of Python, but using pyecharts to draw the chart, pay attention to the matching of the Python version. pyecharts v1.0.0 stops supporting and maintaining Python2.7, 3.4~3.5 versions, and only supports Python3.6+.

Traceback (most recent call last):
  File "*******", line 11, in <module>
    from pyecharts import options as opts
  File "/usr/local/lib/python2.7/site-packages/pyecharts/__init__.py", line 1, in <module>
    from pyecharts import charts, commons, components, datasets, options, render, scaffold
  File "/usr/local/lib/python2.7/site-packages/pyecharts/charts/__init__.py", line 2, in <module>
    from ..charts.basic_charts.bar import Bar
  File "/usr/local/lib/python2.7/site-packages/pyecharts/charts/basic_charts/bar.py", line 17
    series_name: str,
               ^
SyntaxError: invalid syntax

9. The problem of incomplete image display¶

During the test, it was found that the CPU and memory data were saved in sequence, and one of the pictures may be missing, and only half of the image content was displayed. In matplotlib, the position of the axis Axes is specified in standardized graphic coordinates. It may happen that the axis label, title, tick label, etc. will exceed the graphic area, resulting in incomplete display. After adding tight_layout adaptive call, the problem is fixed. tight_layout will automatically adjust the sub-picture parameters to fill the entire image area.

from: https://blog.csdn.net/weixin_46931877/article/details/118079843