• How to listen to https with ListenHTTP in NiFi

    31 Dec 2015

    NiFi provides a way of listening for HTTP requests on a port on a NiFi node. The ListenHTTP processor feeds the content of the request as a FlowFile into the rest of the flow. By default it provides a plain text HTTP service. However, you can also configure the processor to provide an SSL endpoint.In order to configure the SSL endpoint, we need to provide a certificate for the server. In NiFi this is provided by a controller service, in this case StandardSSLContextService.

    When configuring the SSL Service we need to provide a keystore and truststore. These can be created with the Java keytool.

    #!/bin/sh
    
    KEYPASS=changeit
    STOREPASS=changeit
    
    echo "Generate server certificate and export it"
    ${JAVA_HOME}/bin/keytool -genkey -alias server-alias -keyalg RSA -keypass $KEYPASS -storepass $STOREPASS -keystore keystore.jks
    ${JAVA_HOME}/bin/keytool -export -alias server-alias -storepass $STOREPASS -file server.cer -keystore keystore.jks
    
    echo "Create trust store"
    ${JAVA_HOME}/bin/keytool -import -v -trustcacerts -alias server-alias -file server.cer -keystore cacerts.jks -keypass $KEYPASS -storepass $STOREPASS
    

    This will yield two key files, cacerts.jks and keystore.jks (and server.crt, which is not strictly necessary beyond this step, but contains an exported version of the server certificate).

    Note that the details requested when creating the server certificate form the DN, or distinguishing name of the server certificate. Since the keystore can in theory contain multiple certificates, NiFi will need the DN to determine which certificate is used.

    Once we have the two files, we can setup the controller service:

    Now we have a service we can give to the ListenHTTP processor, and a DN. Set these properties, and we have a fully SSL encrypted service listening for POST requests. Of course in this instance we are not using a certificate signed by a proper authority, so we can either have our client use the same cacerts file we produced here, or our course get a more properly trusted certificate.

    comments powered by Disqus
  • Ambari only serves gzip

    23 Feb 2015

    By default Ambari only serves up gzip encoded resources. This is of course the right thing to do. However, sometimes the realities of a corportate network mean old proxies that do odd things. This will stop Ambari from working, and frankly makes it look really weird (just like any SPA without its styles and scripts).

    To enable Ambari to host out plain encoded versions as well, just run:

    for a in /usr/lib/ambari-server/web/{javascripts,stylesheets}/*.gz; do gzip -dc $a > ${a%.gz}; done
    

    on your ambari-server. This will uncompress the gzip files to generate plain text encoded versions of them which Ambari’s spring content negotiation will then serve out in the absence of gzip in the Accept-Encoding header. Problem solved.

    See my stack overflow post on this

    more…
  • Machine Learning without the PhD in the Cloud with Azure ML

    19 Sep 2014

    This talk was give at CloudBurst 2014 in Stockholm, Sweden.

    I promise to put some more info up here later!

    Slides

    more…
  • NoSQL Matters Dublin 2014

  • Getting your Big Data on with HDInsight

  • Know your data lineage

  • Riding the Elephant - Hadoop 2.0

  • Finding (and using!) Big Data in your business

  • When to NoSQL and when to know SQL

  • See the archive for more posts

StackOverflow Flair

profile for Simon Elliston Ball at Stack Overflow, Q&A for professional and enthusiast programmers