Local variable 'sql' referenced before assignment

satvik · November 30, 2022, 5:49pm

I get the following error :- local variable ‘sql’ referenced before assignment when I try to import a file using import a small local file option in hue. We are currently using presto with hive meta-store and Mariadb on a Kubernetes Environment . With hue UI to access the presto . Any anyone please let me know what can be the issue ? I suspect there is misconfiguration with how hue is configured . Below is the hue configuration . Can anyone please let me know if anything more needs to be configured

    hue:
database:
  create: false
  persist: true
  engine: "mysql"
  host: "sip-presto-mariadb"
  port: 3306
  user: "hue"
  password: "hue"
  name: "hue"
interpreters: |
  [[[mysql]]]
  name = Mariadb
  interface=sqlalchemy
  options='{"url": "mysql://hue:hue@sip-presto-mariadb:3306/hue"}'

  [[[presto]]]
  name = Presto
  interface=sqlalchemy
  options='{"url": "presto://sip-presto-coordinator:80/hive/default"}'

  # Set a particular S3 bucket as the default
  [filebrowser]
  remote_storage_home=s3a://bucket_name

ini: |
  [desktop]
  secret_key=hue123
  app_blacklist=search,hbase,security,jobbrowser,oozie,zookeeper,sqoop,pig,impala
  django_debug_mode=false
  gunicorn_work_class=sync
  enable_prometheus=true
  [[task_server]]
  enabled=false
  [[session]]
  trusted_origins=.synchronoss.net
  
  [notebook]
  enable_notebook_2=true

bjorn_alm · December 7, 2022, 1:42pm

Hi again. I’m not sure why this is happening. Can you attach any related messages in the log files so that we can see where this is happening?

satvik · December 8, 2022, 6:14pm

@bjorn_alm
I see the below error stacktrace when I try to import any file using small local file option
Note that We are adding hue user to MariaDb similar to the link :- https://docs.gethue.com/administrator/administration/database/#mysql with the help of configmap

[08/Dec/2022 03:52:20 -0800] middleware   WARNING  request.fs was not set for anonymous user
[08/Dec/2022 03:52:20 -0800] access       INFO     10.223.204.70 -anon- - "GET /desktop/debug/is_alive HTTP/1.1" returned in 11ms 200 0
[08/Dec/2022 03:52:20 -0800] access       INFO     10.223.204.70 -anon- - "GET /desktop/debug/is_alive HTTP/1.1" returned in 11ms 200 0
10.223.204.70 - - [08/Dec/2022:03:52:20 -0800] "GET /desktop/debug/is_alive HTTP/1.1" 200 0 "-" "kube-probe/1.21"
[2022-12-08 03:52:20 -0800] [90] [DEBUG] Closing connection.
[2022-12-08 03:52:21 -0800] [90] [DEBUG] POST /api/editor/autocomplete/default/ticket_small_xlsx/
[08/Dec/2022 03:52:21 -0800] decorators   ERROR    Error running autocomplete
Traceback (most recent call last):
  File "/usr/share/hue/build/env/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
    self.dialect.do_execute(
  File "/usr/share/hue/build/env/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/share/hue/build/env/lib/python3.8/site-packages/MySQLdb/cursors.py", line 209, in execute
    res = self._query(query)
  File "/usr/share/hue/build/env/lib/python3.8/site-packages/MySQLdb/cursors.py", line 315, in _query
    db.query(q)
  File "/usr/share/hue/build/env/lib/python3.8/site-packages/MySQLdb/connections.py", line 239, in query
    _mysql.connection.query(self, query)
MySQLdb._exceptions.OperationalError: (1142, "SHOW command denied to user 'hue'@'11.11.11.11' for table 'ticket_small_xlsx'")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/share/hue/desktop/libs/notebook/src/notebook/decorators.py", line 119, in wrapper
    return f(*args, **kwargs)
  File "/usr/share/hue/desktop/libs/notebook/src/notebook/api.py", line 754, in autocomplete
    autocomplete_data = get_api(request, snippet).autocomplete(snippet, database, table, column, nested, action)
  File "/usr/share/hue/desktop/libs/notebook/src/notebook/connectors/sql_alchemy.py", line 105, in decorator
    raise e
  File "/usr/share/hue/desktop/libs/notebook/src/notebook/connectors/sql_alchemy.py", line 99, in decorator
    return func(*args, **kwargs)
  File "/usr/share/hue/desktop/libs/notebook/src/notebook/connectors/sql_alchemy.py", line 453, in autocomplete
    columns = assist.get_columns(database, table)
  File "/usr/share/hue/desktop/libs/notebook/src/notebook/connectors/sql_alchemy.py", line 561, in get_columns
    return self.db.get_columns(table, database)
  File "/usr/share/hue/build/env/lib/python3.8/site-packages/sqlalchemy/engine/reflection.py", line 372, in get_columns
    col_defs = self.dialect.get_columns(
  File "<string>", line 2, in get_columns
  File "/usr/share/hue/build/env/lib/python3.8/site-packages/sqlalchemy/engine/reflection.py", line 56, in cache
    ret = fn(self, con, *args, **kw)
  File "/usr/share/hue/build/env/lib/python3.8/site-packages/sqlalchemy/dialects/mysql/base.py", line 2505, in get_columns
    parsed_state = self._parsed_state_or_create(
  File "/usr/share/hue/build/env/lib/python3.8/site-packages/sqlalchemy/dialects/mysql/base.py", line 2741, in _parsed_state_or_create
    return self._setup_parser(
  File "<string>", line 2, in _setup_parser
  File "/usr/share/hue/build/env/lib/python3.8/site-packages/sqlalchemy/engine/reflection.py", line 56, in cache
    ret = fn(self, con, *args, **kw)
  File "/usr/share/hue/build/env/lib/python3.8/site-packages/sqlalchemy/dialects/mysql/base.py", line 2772, in _setup_parser
    sql = self._show_create_table(
  File "/usr/share/hue/build/env/lib/python3.8/site-packages/sqlalchemy/dialects/mysql/base.py", line 2871, in _show_create_table
    rp = connection.execution_options(
  File "/usr/share/hue/build/env/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2179, in execute
    return connection.execute(statement, *multiparams, **params)
  File "/usr/share/hue/build/env/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 982, in execute
    return self._execute_text(object_, multiparams, params)
  File "/usr/share/hue/build/env/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1149, in _execute_text
    ret = self._execute_context(
  File "/usr/share/hue/build/env/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1252, in _execute_context
    self._handle_dbapi_exception(
  File "/usr/share/hue/build/env/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1473, in _handle_dbapi_exception
    util.raise_from_cause(sqlalchemy_exception, exc_info)
  File "/usr/share/hue/build/env/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/share/hue/build/env/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 152, in reraise
    raise value.with_traceback(tb)
  File "/usr/share/hue/build/env/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
    self.dialect.do_execute(
  File "/usr/share/hue/build/env/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/share/hue/build/env/lib/python3.8/site-packages/MySQLdb/cursors.py", line 209, in execute
    res = self._query(query)
  File "/usr/share/hue/build/env/lib/python3.8/site-packages/MySQLdb/cursors.py", line 315, in _query
    db.query(q)
  File "/usr/share/hue/build/env/lib/python3.8/site-packages/MySQLdb/connections.py", line 239, in query
    _mysql.connection.query(self, query)
sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (1142, "SHOW command denied to user 'hue'@'11.11.11.11' for table 'ticket_small_xlsx'")
[SQL: SHOW CREATE TABLE `default`.`ticket_small_xlsx`]

Also from this link :- https://docs.gethue.com/administrator/administration/database/#mysql
I see that mysqlclient needs to be installed for the mysql dialect to be added to the Python system or Hue Python virtual environment . As of now we only mentioning as interface=sqlalchemy in the values file that is already provided above . Could you let me know how this mysqlclient needs to be installed in a kubernetes environment or just mentioning as interface=sqlalchemy suffices .

bjorn_alm · December 8, 2022, 7:30pm

@Harsh Can you take a look at this?