Encountered java.net.UnknownHostException

[jerryleooo]

presto:sample> select count(1) from viewlog;

Query 20180130_203003_00003_umfi8, FAILED, 1 node
Splits: 17 total, 0 done (0.00%)
0:05 [0 rows, 0B] [0 rows/s, 0B/s]

Query 20180130_203003_00003_umfi8 failed: java.net.UnknownHostException: scluster

Presto-Admin version: 2.3
Presto version: 0.192

This is the content of hive.properties:

connector.name=hive-hadoop2
hive.metastore.uri=thrift://hadoop-thriftserver:9083
hive.config.resources=/home/work/hadoop/etc/hadoop/core-site.xml,/home/work/hadoop/etc/hadoop/hdfs-site.xml

Before I use presto-admin, I have already setup presto on our 3-nodes cluster successfully. As the machines become more and more, I consider using presto-admin but it seems not work.

I copied the configuration from previous presto installation.

Then I manually started the cluster (find the launcher file and start it) instead of using presto-admin server start, the output of the query is a little different:

presto:sample> select count(1) from viewlog;

Query 20180130_203809_00000_wj3th, FAILED, 3 nodes
Splits: 1,791 total, 1,635 done (91.29%)
0:21 [661M rows, 90.9GB] [31.1M rows/s, 4.28GB/s]

Query 20180130_203809_00000_wj3th failed: Error opening Hive split hdfs://scluster/products/viewlog/cooked/grass_region=VN/grass_date=2018-01-10/part-00008-944633fd-e596-4067-8532-aa0f19030691.snappy.parquet (offset=67108864, length=67108864): java.net.UnknownHostException: scluster

To make things simple, there is no other workers than the worker on the coordinator machine.

Any ideas?

[jerryleooo]

I may know the reason: it is related with different users in two installations.

I use user work before tried presto-admin, and presto-admin seems using root? I find this when I tried run the same hdfs command by different users:

[root@sample presto-client-0.192]# hadoop fs -ls hdfs://scluster
ls: `hdfs://scluster': No such file or directory
[root@sample presto-client-0.192]#

work@sample:/usr/lib/presto
$ hadoop fs -ls hdfs://scluster
Found 9 items
drwx------   - work supergroup          0 2018-01-30 23:50 hdfs://scluster/user/work/.Trash
drwxr-xr-x   - work supergroup          0 2018-01-31 00:30 hdfs://scluster/user/work/.sparkStaging

[kokosing]

Please use different user than root in your hadoop configuration, you can change it by following this: https://prestodb.io/presto-admin/docs/current/installation/presto-admin-configuration.html

Presto related part of the doc is here: https://prestodb.io/docs/current/connector/hive.html#hdfs-username

I am going to close this, in case it does not solve your issue, please reopen.

[aaronhadoop]

I solve this question by the following step:

1. copy the hadoop installation directory to the presto node;
2. config the ${PRESTO_HOME}/etc/catalog/hive.properties (hive.config.resources=${HADOOP_HOME}/etc/hadoop/core-site.xml, ${HADOOP_HOME}/etc/hadoop/hdfs-site.xml)

maybe you can read this document (https://yujianxin.blog.csdn.net/article/details/122391796)