I have used ‘bigquery’ connector for submitting query into bigquery.
submitting works. but I cannot see metadata of the bigquery tables.
is there any solution to see the tables and data set in bigquery in hue ??
Databases are actually BigQuery datasets. Are you sure that your project has some datasets created? (can check via the BigQuery UI and expand it in the Resource section).
Note: currently you need to put explicitly the dataset name to be able to browse the tables
"url": "bigquery://projectName/datasetName"
But this will work soon:
"url": "bigquery://projectName"I’m sorry I asked it in confusing way.
I want to see dataset of bigquery in left panel.
Like Borwser, I need to click and expand resources of bigquery and want to see metadata on it.
is it supported?
All the datasets of your project will be listed as databases. It does not list other projects datasets yet (e.g. bigquery-public-data).
Do you have a screenshot of the Big Query console and the interpreter config you used?
Are you sure that your project contains datasets?
In left panel, there aren’t no namespace or metadata.
Isn’t there any way to get schema and dataset from bigquery?
(it seems like there are dataset list in the right small slot but, not enough information…)
Which Hue version is it? “No namespaces” is probably an unrelated bug. Any change to upgrade to the very latest? (4.5 http://gethue.com/hue-4-5-and-its-improvements-are-out/)
this is latest 4.5 version. I got serveral errors but not related namespace.
[13/Aug/2019 18:20:32 +0000] views ERROR Could not create home directory at login for youngin1.lee.
Traceback (most recent call last):
File “/home/bxb_anl/youngin/hue/hue/desktop/core/src/desktop/auth/views.py”, line 136, in dt_login
ensure_home_directory(request.fs, user)
File “/home/bxb_anl/youngin/hue/hue/apps/useradmin/src/useradmin/views.py”, line 820, in ensure_home_directory
fs.do_as_user(username, fs.create_home_dir, home_directory)
File “/home/bxb_anl/youngin/hue/hue/desktop/core/src/desktop/lib/fs/proxyfs.py”, line 103, in do_as_user
return fn(*args, **kwargs)
File “/home/bxb_anl/youngin/hue/hue/desktop/core/src/desktop/lib/fs/proxyfs.py”, line 172, in create_home_dir
self._get_fs(home_path).create_home_dir(home_path)
File “/home/bxb_anl/youngin/hue/hue/desktop/libs/hadoop/src/hadoop/fs/hadoopfs.py”, line 172, in create_home_dir
if not self.exists(home_path):
File “/home/bxb_anl/youngin/hue/hue/desktop/libs/hadoop/src/hadoop/fs/webhdfs.py”, line 302, in exists
return self._stats(path) is not None
File “/home/bxb_anl/youngin/hue/hue/desktop/libs/hadoop/src/hadoop/fs/webhdfs.py”, line 290, in _stats
raise ex
WebHdfsException: HTTPConnectionPool(host=‘localhost’, port=50070): Max retries exceeded with url: /webhdfs/v1/user/youngin1.lee?op=GETFILESTATUS&user.name=hue&doas=youngin1.lee (Caused by NewConnectionError(’<urllib3.connection.HTTPConnection object at 0x7f909814d850>: Failed to establish a new connection: [Errno 111] Connection refused’,))
[13/Aug/2019 18:20:37 +0000] exceptions_renderable ERROR Potential trace: [(’/home/bxb_anl/youngin/hue/hue/apps/jobbrowser/src/jobbrowser/views.py’, 147, ‘jobs’, ‘jobs = get_api(request.user, request.jt).get_jobs(’), (’/home/bxb_anl/youngin/hue/hue/apps/jobbrowser/src/jobbrowser/api.py’, 48, ‘get_api’, ‘return YarnApi(user)’), (’/home/bxb_anl/youngin/hue/hue/apps/jobbrowser/src/jobbrowser/api.py’, 78, ‘init’, ‘self.resource_manager_api = resource_manager_api.get_resource_manager(user.username)’), (’/home/bxb_anl/youngin/hue/hue/desktop/libs/hadoop/src/hadoop/yarn/resource_manager_api.py’, 52, ‘get_resource_manager’, “raise PopupException(_(‘No Resource Manager are available.’))”)]
[13/Aug/2019 18:20:37 +0000] middleware INFO Processing exception: No Resource Manager are available.: Traceback (most recent call last):
File “/home/bxb_anl/youngin/hue/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/handlers/base.py”, line 185, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File “/home/bxb_anl/youngin/hue/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/utils/decorators.py”, line 185, in inner
return func(*args, **kwargs)
File “/home/bxb_anl/youngin/hue/hue/apps/jobbrowser/src/jobbrowser/views.py”, line 164, in jobs
raise PopupException(ex)
PopupException: No Resource Manager are available.
Those are because Hue is not configured to point to Hadoop HDFS and YARN and assumes they are on localhost. This will be fixed with https://issues.cloudera.org/browse/HUE-8758.
In the meantime, having an empty hadoop section in the hue.ini should fix it:
[hadoop]
Ok I can let [hadoop] having empty, but,
I just want to see the datasets and schema of bigquery on the left panel.
isn’t there any way to do that?
I solved like this.
options=’{“url”: “bigquery://analytics-dev”, “credentials_path”: “/home/youngin1.lee/analytics-dev-4b71ccf51b90.json”}’
before, I just included credential in the syetem path.