CSV export with non-unicode data

#1

When exporting CSV files using HUE we sometimes get the following error:

image
image1427×436 172 KB

It is probably related to a field that is custom filled by the users, so weird characters are possible. However, there are no problems with any of such weird characters in Hive or any of the other databases. Additionally, the export straight to Excel (on HUE) works without issues too.

It seems the problem is specifically with the CSV exporter used by HUE.

Is there a quick fix for that? While researching the issue, plenty of various fixes are suggested, however it’s not clear for me exactly where the issue arises here in HUE.

#2

IIRC there is a potential issue with tablib.

Does this work for you or not?

CREATE table a (a string);

insert into a values ('prêté serment '), ('的');

SELECT * from a;

We could investigate first by upgrading https://pypi.org/project/tablib/ and its dependencies

1 Like
#3

Note: “Due to tablib upgrade for Hue in CDP from 0.10.x to 0.14.x release which is generally used for Python3 release, has made CSV download which has some special char in the dataset to fail with the above error message.”

1 Like
#4

If using Python 3 we should try with the latest tablib 3.

Note: works for me on Python 3: https://demo.gethue.com

1 Like
#5

Thanks, @Romain, downgrading to 0.10.0 resolved that specific error. Are there any downsides regarding the downgrade, i.e., some potential issues that may arise down the road, or is downgrading a good solution for HUE?

We’re working on a Centos 7 machine, so Python 2.7 is still the default. Not sure that a global upgrade to Python 3 would be a good idea, since some other older services might still depend on it. On the other hand, maybe it’s possible to only migrate Hue locally to Python 3 (unless Python 2 is perfectly fine)?

#6

downgrading to 0.10.0 is recommended as pretty contained while the Python3 Hue is working well so far but is a bigger change (and some pieces might still need to be ironed out a bit, but we also welcome to do it as this is where the future efforts are landing).