Announcing: MEP 2.0 Released

Document created by Rachel Silver Employee on Dec 6, 2016Last modified by wochanda on Jan 24, 2017
Version 15Show Document
  • View in full screen mode

Announcing MapR Ecosystem Pack (MEP) 2.0!

Date: 12/09/16

 

We’re pleased to announce the general release of the MapR Ecosystem Pack (MEP) version 2.0. This represents the second major release of a MapR Ecosystem Pack since the beginning of this new process of delivering ecosystem upgrades.

 

If you’re new to this process, MapR Ecosystem Packs are a way to deliver ecosystem upgrades decoupled from core upgrades - allowing you to upgrade your tooling independently of your MapR Converged Data Platform.

 

For more information about the MEP process, please see our post on the MapR Ecosystem Packs Process here:
MapR Ecosystem Packs Process 

 

MEP 2.0 contains a series of important upgrades and new features:

 

Upgrades 

Spark 2.0.1 GA

We had previously released a Spark 2.0 Developer Preview in June, and now we’re proud to release the MapR Spark 2.0.1 GA release as part of MEP 2.0.

 

Key improvements: 

 

Structured APIs:

  • Runs on the same engine as SparkSQL.
  • Allows access to data from a variety of different data sources.
  • Can run database-like operations or allow for passing in custom code.

 

Spark as a Compiler:

  • Whole-stage code generation is provided by the second-generation Tungsten engine.
  • Eliminates the need for multiple JVM calls by flattening SQL queries into one single function evaluated as bytecode at runtime.

 

Note that the exciting Structured Streaming feature, which provides a tabular view into streaming data, is still an alpha release by the community, and thus the APIs are still experimental. Stay tuned for updates on this feature as it becomes GA and subsequently supported in the MapR Platform.

 

Drill 1.9

Drill 1.9 is an iterative release from the Apache Drill community and is now available on MapR with the MEP 2.0 release.

 

The key highlights of this release include:

  • Enhanced Parquet Performance - Improved query performance for I/O intensive analytic queries using an optimized Parquet reader, as well as significant performance boosts for targeted queries by reducing I/O via Parquet filter pushdown and Limit operator pushdown. These techniques complement the variety of other Drill optimizations, including partition pruning and metadata caching to further enhance the performance.
  • Flexible and Dynamic UDFs - Enables data scientists, analysts, and developers to develop and deploy custom Drill SQL functions (UDFs) in a self-service fashion without having to restart Drill services in the cluster or require IT involvement. This feature is greatly useful in large, multi-tenant organizations where restarting Drill services is disruptive to users. The feature also empowers users to get fast value from data using Apache Drill
  • Seamless BI tool integration - In this release, Drill introduces a variety of SQL improvements to enable optimal BI tool integration. This includes support for a variety of join syntax generated from Tableau and other BI tools, as well as improvements to the number of the queries generated for metadata from the BI tools. These enhancements improve the overall interactive user experience.

 

Hue 3.10

Hue 3.10 has provided the following improvements:

  • Oozie improvements
    • External Workflow Graph
    • Single Action Execution
    • New Ability: Dryrun Oozie job
  • New SQL Query Editor works over JDBC
    • Look for an upcoming Community post on how to use this with Apache Drill!
  • Directory and file-based document management
    • Users can create their own directories and subdirectories and drag and drop documents within the simple file browser interface

 

 

New Components

 

MapR Installer Stanzas

MapR Installer Stanzas enable API-driven installation. These provide the ability to build a configuration file called a “stanza” which contains layout and settings for a cluster installation that can be passed programmatically to the installer.

 

Kafka Connect for MapR Streams

Kafka Connect for MapR Streams is a new way to easily connect common data systems with Kafka by providing prebuilt connectors for legacy and modern data stores.

 

Kafka REST Proxy for MapR Streams

Kafka REST Proxy for MapR Streams provides the ability for any device that can communicate using HTTP to easily publish/subscribe to Kafka topics.

 

MapR Teradata Connector (powered by Teradata Connector for Hadoop)

In partnership with Teradata, we're introducing the Teradata Connector for MapR, a MapR implementation of the Teradata Connector for Hadoop (TDCH). This is a Sqoop wrapper, built into MapR Sqoop, that facilitates bulk data transfer between Hadoop and external data storage.

 

 

All Components (* denotes re-release)

The following is a list of components included in the MEP 1.0 release, supported for MapR 5.2.

 

MEP 2.0 Contents
Release NotesDocumentation
Apache Drill 1.9Release NotesDocumentation
Apache Hive 1.2.1*Release NotesDocumentation
Apache Flume 1.6Release NotesDocumentation
Apache HBase 1.1.1Release NotesDocumentation
AsyncHBase 1.7Release NotesDocumentation
Apache Mahout 0.12.0*Release NotesDocumentation
Apache Myriad 0.1.0Release NotesDocumentation
Apache Oozie 4.2.0*Release NotesDocumentation

Apache Pig 0.16

Release NotesDocumentation
Apache Sentry 1.6Release NotesDocumentation
Apache Spark 2.0.1Release NotesDocumentation
Apache Sqoop 1.4.6*Release NotesDocumentation
Apache Sqoop2 1.99.7Release NotesDocumentation
Apache Storm 0.10.0*Release NotesDocumentation
HttpFS 1.0Release NotesDocumentation
Hue 3.10Release NotesDocumentation
Impala 2.5Release NotesDocumentation
Kafka Connect for MapR StreamsRelease NotesDocumentation
Kafka REST Proxy for MapR StreamsRelease NotesDocumentation
MapR Installer StanzasRelease NotesDocumentation
MapR Teradata Connector (Powered by TDCH)Release Notes (Sqoop)Documentation

 

Download

MEP 2.0.0:

Index of /releases/MEP/MEP-2.0.0 

 

UI Installer:

Index of /releases/installer 

 

Documentation

Have a Question?

Ask in Answers or comment below.

 

Converge Community Resources

drill mep spark mapr installer kafka

1 person found this helpful

Attachments

    Outcomes