How to upgrade Hive metastore using Hive schema tool

Document created by Hao Zhu Employee on Feb 17, 2016
Version 1Show Document
  • View in full screen mode

Author: Hao Zhu

Original Publication Date: November 21, 2014

 

Starting from Hive 0.12, Hive schema tool is introduced for Hive metastore schema manipulation. It can have below 3 usage:

 

1. Initialize the metastore schema for the current Hive version.
2. Verify if the metastore schema version is compatible with Hive binaries.
3. Upgrade the schema from an older version to current.

 

Before Hive 0.12, we have to manually run the upgrade DDL SQL files in the metastore database.This article will go through the steps to upgrade metastore schema using Hive schema tool, taking PostgreSQL for example.

 

Hive 0.12 introduced a new table "VERSION" in Hive metastore, that is why Hive can recognize the metastore version.

 

So upgrading from 0.12 to newer version, Hive schema tool knows which upgrade SQL files to execute.If we plan to upgrade from an older version, we must know what is current version of metastore because Hive schema tool can never know by itself.

 

Here are 2 examples, one is upgrading metastore from 0.12 from current 0.13, and the other one is upgrading from 0.8 to 0.13.

Upgrading metastore schema from 0.12 to 0.13.

1. Verify current versions of Hive binary and Hive metastore.

[root@n1a conf]#  schematool -dbType postgres -info 

Metastore connection URL: jdbc:postgresql://n1a/oldmetastore2

Metastore Connection Driver : org.postgresql.Driver

Metastore connection User: hiveuser

Hive distribution version: 0.13.0

Metastore schema version: 0.12.0

schemaTool completeted

 

2. Dry run can let us know what SQL file to execute in advance.

[root@n1a conf]# schematool -dbType postgres -upgradeSchema -dryRun

Metastore connection URL: jdbc:postgresql://n1a/oldmetastore2

Metastore Connection Driver : org.postgresql.Driver

Metastore connection User: hiveuser

Starting upgrade metastore schema from version 0.12.0 to 0.13.0

Upgrade script upgrade-0.12.0-to-0.13.0.postgres.sql

schemaTool completeted

3. Execute the upgrade.

[root@n1a conf]# schematool -dbType postgres -upgradeSchema

Metastore connection URL: jdbc:postgresql://n1a/oldmetastore2

Metastore Connection Driver : org.postgresql.Driver

Metastore connection User: hiveuser

Starting upgrade metastore schema from version 0.12.0 to 0.13.0

Upgrade script upgrade-0.12.0-to-0.13.0.postgres.sql

Completed upgrade-0.12.0-to-0.13.0.postgres.sql

schemaTool completeted

4. Verify.

[root@n1a conf]#  schematool -dbType postgres -info

Metastore connection URL: jdbc:postgresql://n1a/oldmetastore2

Metastore Connection Driver : org.postgresql.Driver

Metastore connection User: hiveuser

Hive distribution version: 0.13.0

Metastore schema version: 0.13.0

schemaTool completeted

Attachments

    Outcomes