The root scratch dir: /tmp/hive on HDFS should be writable. Current permissions are: rw-rw-rw-

Recently when I run Spark + HDFS on Win7 desktop. I always got following error message in Spark's driver log.

java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable. Current permissions are: rw-rw-rw-

Somebody suggested if I can run winutils ls /tmp/hive in Win7. I tried, and got this error.

access denied or FindFileOwnerAndPermission error (1789): The trust relationship between this workstation and the primary domain failed.

It seems something was wrong in my local Win7. I struggled for a time, and finally found the reason. Recently I modified my DNS of local network connection, and provided a wrong IP address. Winutils just could not connect to our Domain Controller for user verification. Once I set up a correct DNS of my network, and restart the machine, the issue was gone!

Show Chinese Characters in Matplotlib Plot on Mac-OS

Mac-OS uses different font file from Windows. If a string containing Chinese characters is show in a chart of matplatlib, the Chinese character will be shown as [][]. And there is a warning message:

UserWarning: findfont: Font family [u'sans-serif'] not found. Falling back to Bitstream Vera Sans (prop.get_family(), self.defaultFamily[fontext]))

Following steps can solve this issue:

1. Download Chinese font simhei.ttf from http://www.font5.com.cn/font_download.php?id=151&part=1237887120
2. Extract the file RAR file and copy simhei.ttf to anaconda3/lib/python3.7/site-packages/matplotlib/mpl-data/fonts/ttf.
3. Edit anaconda3/lib/python3.7/site-packages/matplotlib/mpl-data/matplotlibrc. Find sans-serif. Uncomment this line, and save the change.
4. Remove all files and directories under ~/.matplotlib.
5. Restart jupyter notebook.
6. In Python programe, add following lines before plot:

        from pylab import mpl
        mpl.rcParams['font.sans-serif'] = ['SimHei']
        mpl.rcParams['axes.unicode_minus'] = False
     
Okay, Chinese characters can be displayed correctly in chart.