Senior Big Data Engineer | Remote Position - Building Enterprise Data Infrastructure with Hadoop, Spark, Python & Cloud Technologies

---

About arenaflex

Welcome to arenaflex, where innovation meets opportunity and data drives the future of financial services. At arenaflex, we believe in empowering individuals and businesses through cutting-edge technology and unparalleled expertise. As a global leader in the financial industry, we are committed to transforming the way people interact with money, investments, and financial services.

Our mission is simple yet profound: to deliver exceptional value to our customers while fostering an environment where talented professionals can thrive, grow, and make meaningful contributions. We pride ourselves on maintaining a culture of innovation, collaboration, and continuous learning. Our team members are our greatest asset, and we invest heavily in their development, well-being, and career advancement.

At arenaflex, we embrace the power of data to make smarter decisions, create better products, and deliver outstanding customer experiences. Our technology infrastructure supports millions of transactions daily, and we are constantly pushing the boundaries of what's possible with big data, machine learning, and cloud computing. Join us on our journey to revolutionize the financial industry through technology excellence.

About the Role

We are seeking a talented and passionate Senior Big Data Engineer to join our dynamic technology team at arenaflex. This is a remote position that offers the flexibility to work from anywhere while contributing to meaningful projects that impact millions of customers worldwide.

As a Senior Big Data Engineer at arenaflex, you will be an integral part of our fast-paced, innovative engineering team. You will work on challenging problems involving massive-scale data processing, real-time analytics, and advanced machine learning applications. Your expertise will help shape the future of our data infrastructure and enable data-driven decision-making across the organization.

This role offers an exciting opportunity to work with cutting-edge technologies including Hadoop, Spark, Python, SQL, and various cloud platforms. You will collaborate with talented engineers, data scientists, and product managers to deliver innovative solutions that power arenaflex's products and services.

Key Responsibilities

As a Senior Big Data Engineer, you will be responsible for designing, developing, and maintaining our enterprise-level data infrastructure. Your day-to-day responsibilities will include:

• Designing and Building Data Pipelines: Create robust, scalable data pipelines for both batch and real-time processing using technologies like Hadoop, Spark, Kafka, and cloud-native services. Ensure data flows seamlessly between various systems and platforms while maintaining data quality and integrity.
• Developing Big Data Solutions: Write efficient, maintainable code in Python, Scala, or Java to process large datasets. Implement data transformation, aggregation, and enrichment processes that support business intelligence and analytical requirements.
• Architecting Data Systems: Design and implement scalable data architectures that can handle petabytes of data while maintaining performance and reliability. Work with stakeholders to understand data requirements and translate them into technical solutions.
• Code Development and Review: Write high-quality code, including ongoing refactoring to improve code structure and quality. Conduct code reviews, provide technical guidance to junior team members, and ensure adherence to best practices and coding standards.
• API Development and Testing: Design and implement APIs for data access and integration. Develop comprehensive test plans, perform unit testing, integration testing, and ensure robust error handling and data validation.
• Performance Optimization: Optimize data processing jobs, queries, and pipelines for performance and efficiency. Identify bottlenecks and implement solutions to improve processing speeds and reduce resource consumption.
• Agile Collaboration: Act as a core member of Agile/Scrum teams, participating in sprint planning, daily stand-ups, and retrospectives. Contribute to user story analysis, elaboration, and estimation.
• Documentation and Knowledge Sharing: Create and maintain technical documentation, including design specifications, API documentation, and operational runbooks. Share knowledge with team members through mentoring and technical sessions.
• Innovation and Improvement: Identify opportunities to adopt new technologies and approaches. Embrace a mindset of continuous improvement by exploring innovative solutions beyond conventional approaches.
• Cross-Functional Partnership: Work closely with data scientists, product managers, and business stakeholders to understand requirements and deliver solutions that meet business needs.

Essential Qualifications

To succeed in this role, you must possess the following qualifications and experience:

• Educational Background: Bachelor's degree or higher in Computer Science, Information Systems, Engineering, or a related technical field. A Master's degree in Computer Applications or equivalent experience is preferred.
• Professional Experience: Minimum of 2+ years of programming development experience with demonstrated leadership in guiding teams of developers and facilitating Scrum teams. You should have a proven track record of delivering complex technical projects.
• Big Data Technologies: Hands-on experience with MapReduce, Hive, and Spark (core, SQL, and PySpark). Strong understanding of distributed computing principles and the Hadoop ecosystem is essential.
• Data Warehousing: Demonstrated expertise in data warehousing concepts and methodologies. Experience designing and implementing data models for analytical purposes.
• Programming Skills: Strong proficiency in Core Java, Python, or Scala. Ability to write efficient, maintainable code and understand software design patterns.
• Hadoop and Spark Architecture: Mastery of Hadoop and Spark architecture along with their working principles. Understanding of how these technologies integrate and scale within enterprise environments.
• SQL and Data Processing: Hands-on experience writing and optimizing complex SQL queries using Hive and PySpark DataFrames. Skills in optimizing joins while processing massive datasets efficiently.
• Unix/Linux Skills: Experience with Unix shell scripting for automation and data processing tasks. Familiarity with Linux environments and command-line tools.
• Data Pipeline Development: Ability to design and develop advanced data pipelines for both batch and real-time data processing. Understanding of various data processing patterns and architectures.
• SDLC Experience: Experience in analysis, design, development, testing, and implementation of enterprise applications. Familiarity with software development methodologies and best practices.
• Technical Documentation: Demonstrated ability to develop and document technical and functional specifications. Strong analytical skills to troubleshoot programming and system processing flows.

Preferred Qualifications

While the following qualifications are not mandatory, they will significantly enhance your candidacy:

• Cloud Platforms: Knowledge of cloud platforms such as GCP (Google Cloud Platform) or AWS (Amazon Web Services). Experience building microservices and scalable solutions in cloud environments is a significant advantage.
• Stream Processing: 2+ years of experience designing and building solutions using Kafka streams or other message queuing systems. Understanding of event-driven architectures is highly valued.
• DevOps Practices: Experience with GitHub or Bitbucket and familiarity with CI/CD (Continuous Integration/Continuous Deployment) pipelines. Understanding of automation and infrastructure as code.
• NoSQL Databases: Experience with NoSQL databases such as HBase, Couchbase, or MongoDB. Understanding of when to use NoSQL vs. relational databases.
• Financial Services Background: Knowledge of financial reporting ecosystems and understanding of regulatory requirements in the financial services industry.
• Communication Skills: Excellent technical and analytical capabilities combined with strong interpersonal skills. Ability to communicate complex technical concepts to non-technical stakeholders.
• Project Management: Strong project management skills with the ability to prioritize tasks, manage deadlines, and deliver results in a fast-paced environment.

Skills and Competencies

Beyond technical qualifications, we are looking for candidates who demonstrate the following competencies:

• Problem-Solving Abilities: Strong analytical skills to approach complex problems methodically and develop innovative solutions. You should be comfortable working with ambiguous requirements and turning them into concrete technical implementations.
• Collaboration and Teamwork: Ability to work effectively in cross-functional teams, share knowledge, and mentor junior team members. You should thrive in a collaborative environment and enjoy learning from others.
• Adaptability: Willingness to learn new technologies and adapt to evolving business requirements. The big data landscape is constantly changing, and we need engineers who embrace change and continuous learning.
• Attention to Detail: Meticulous approach to code quality, testing, and documentation. Small errors can have significant impacts in big data systems, so precision is crucial.
• Ownership Mindset: Take responsibility for your work and deliver high-quality results. We value engineers who take initiative and see projects through to completion.
• Innovation Orientation: A "rebel with a purpose" mindset – someone who looks beyond the obvious for continuous improvement opportunities and isn't afraid to challenge the status quo.

Career Growth and Development

At arenaflex, we are committed to helping our employees grow and advance in their careers. As a Senior Big Data Engineer, you will have numerous opportunities for professional development:

• Technical Leadership: Advance into technical leadership roles where you can guide architectural decisions and mentor other engineers.
• Specialization Paths: Choose to specialize in areas such as real-time streaming, machine learning infrastructure, cloud architecture, or data governance.
• Cross-Functional Exposure: Work on projects spanning multiple business lines, giving you a broad understanding of the financial services industry.
• Training and Certification: Access to comprehensive training programs, certifications, and conferences. We support your pursuit of industry-recognized certifications in big data technologies.
• Career Progression: Clear career pathways to Staff Engineer, Principal Engineer, or Engineering Manager positions based on your skills and aspirations.

Work Environment and Culture

arenaflex offers a supportive and inclusive work environment where diversity is valued and celebrated. Our culture is built on mutual respect, open communication, and a shared commitment to excellence. As a remote employee, you will enjoy:

• Flexible Work Arrangements: Work from the comfort of your home or choose from our modern office spaces. We trust our employees to manage their time effectively and maintain work-life balance.
• Collaborative Tools: Access to state-of-the-art collaboration tools and technologies that enable seamless communication and project management across distributed teams.
• Inclusive Culture: Be part of a diverse team that values different perspectives and experiences. We foster an environment where everyone feels welcome and can contribute their best work.
• Continuous Learning: Access to online learning platforms, technical workshops, and knowledge-sharing sessions with industry experts.
• Employee Wellness: Comprehensive wellness programs and resources to support your physical, mental, and emotional well-being.

Compensation and Benefits

We offer competitive compensation and a comprehensive benefits package designed to reward your contributions and support your well-being:

• Competitive Salary: Base salary range of $20-30 per hour, commensurate with experience and qualifications. We regularly review compensation to ensure we remain competitive in the market.
• Performance Bonuses: Annual performance-based bonuses that recognize your contributions to team and company success.
• Health and Wellness: Comprehensive health insurance coverage including medical, dental, and vision plans for you and your family.
• Retirement Plans: 401(k) retirement savings plan with company matching contributions to help you build financial security for the future.
• Paid Time Off: Generous paid time off policies including vacation, personal days, and holidays to support work-life balance.
• Professional Development: Annual learning and development budget to support certifications, conferences, and training programs.
• Remote Work Perks: Stipends for home office setup, internet connectivity, and wellness expenses.

Join the arenaflex Team

We invite you to join arenaflex and become part of a team that is transforming the financial services industry through technology innovation. This is an exciting opportunity to work with talented engineers, tackle challenging problems, and make a meaningful impact.

At arenaflex, we believe that great ideas can come from anywhere, and we value the unique perspectives and experiences that each team member brings. If you are passionate about data, excited by big data technologies, and ready to take your career to the next level, we want to hear from you.

Take the first step in your journey with arenaflex. Apply today and become part of a team that is shaping the future of financial services through the power of data and technology.

Note: Final offer of employment is contingent upon successful completion of a background check, subject to applicable laws and regulations. We are an equal opportunity employer and encourage applications from all qualified individuals.