No, we haven't used Luigi with Hadoop. For batch processing we use containerized jobs with a simple job queue, like described in the blog article.