Tel: 03 5224 2560
Welcome to Dimond Pony Trading Pty Ltd.!
关闭
Lead Big Data Engineer Over $100,000.00 Collect Apply
Your current location: Home > Position list > Position detail

Lead Big Data Engineer

Annual Salary Over $100,000.00

Professionals - Business and Systems Analysts, and Programmers

NSW - Other Regional (Post code: 2000)

Recruiting numbers: 1

Expiration date: 29-Nov-2024

Full Time
01-Nov-2024 update visited:
Job requirements

Educational requirements: Bachelor

English requirements: Competent English

Requirements for skilled employment experience for years: 1-3 years

Required residence status: Temporary visa, Permanent resident, Citizen

Accept remote work: unacceptable

Mandatory Skills: At least 7 years of overall experience in Big Data Domain with extensive knowledge in Banking, Insurance, and Mortgage Domain. At least 3 years of experience in the Big Data technologies for Batch Implementation using Python, Spark, Hive, Scala, HQL, Hadoop, Phoenix, Hbase, Bash, PowerShell Must have strong experience with DDEP Analytics platform and metadata driven framework Apache ATLAS. Must have experience with code management tools such as Bitbucket, GitHub and SVN. At least 3 years of experience in NoSQL databases such as Hbase, Cassandra and Phoenix. Proficient in Azure cloud services – ADLS2, HD Insight, Azure SQL, Cosmos DB, Blob Storage, Databricks. Must have hands on devops deployment tools – Jenkins, JIRA, Maven, sbt, Azure Devops, docker. Must understand Azure Machine Learning Services and problem-solving skills using Reinforcement, Supervised and Unsupervised learning models. Duties and Responsibilities: As a Lead Big Data Engineer, you will be responsible for designing and implementing the Analytics platform solutions for large and complex projects. You will also be involved in improving the initiative within Cognizant Community that inspire significant improvement across Big Data practices. Work on ingesting, storing, processing, and analyzing large data sets which are useful for business analytics and reporting purposes. Create data governance policies and support data versioning processes to protect customer personal data and help business to adhere security standards. Investigate and analyze alternative solutions to data storing, processing etc. to ensure most streamlined approaches are implemented Translate complex technical and functional requirements into detailed design and document for solution endorsement to commence build and test. Mentoring the team members and guide them to develop business logics using software best practices. Build and refine a robust Machine Learning pipeline to ensure models are responsive to changes and trends in the data. Work closely with a data science team and implementing data analytics pipelines to help business driving the customer solutions to their problems. Work with Platform Architects on the design and deliver new products into both non-production and production clusters along with documentation. Monitor the production jobs and providing support to solve business problems. Create the automation code using automation utilities and help business saving cost by reducing the time efforts. Identify problems and discuss with business teams and propose solutions to solve the identified problems. Optimize Big Data pipelines using tuning performance techniques. Documenting the solutions and explaining the usage to both support team members as well as to the business team. Address and solve complex business problems using large amounts of data as inputs. Managing the Software Development Lifecycle using DevOps pipelines. Code defects fixes that are raised during QT and UAT Testing cycles. Build robust, efficient, and scalable data pipelines to ingest data from different sources into a cloud big data platform. Identify various data risks associated with the data flow process and document themitigation strategies. Cleanse the source raw data and transform and store into cloud cluster database for business analytical purpose.
Contact information
Last online time: 19-Jan-2024 11:28
You can't view contact information,Please send your resume directly Send
The company's other advertisements
  • 1-3 years experience
    Diploma or trade qualification education
    Annual Salary
    $60,000.00 - $80,000.00
    VIC - Bendigo
    22-Nov-2024
    Apply
  • 1-3 years experience
    Diploma or trade qualification education
    Annual Salary
    $60,000.00 - $70,000.00
    SA - Adelaide
    22-Nov-2024
    Apply
  • 1-3 years experience
    Diploma or trade qualification education
    Annual Salary
    $60,000.00 - $80,000.00
    NSW - Sydney
    22-Nov-2024
    Apply
  • 1-3 years experience
    Bachelor education
    Annual Salary
    $60,000.00 - $80,000.00
    WA - Perth
    22-Nov-2024
    Apply
  • 1-3 years experience
    Bachelor education
    Annual Salary
    $60,000.00 - $80,000.00
    QLD - Brisbane
    22-Nov-2024
    Apply
  • 0-1 year experience
    Diploma or trade qualification education
    Annual Salary
    Negotiable
    NSW - Sydney
    22-Nov-2024
    Apply
  • 0-1 year experience
    Diploma or trade qualification education
    Annual Salary
    Negotiable
    NSW - Sydney
    22-Nov-2024
    Apply
  • 0-1 year experience
    Bachelor education
    Annual Salary
    Negotiable
    VIC - Melbourne
    22-Nov-2024
    Apply
  • 3-5 years experience
    Bachelor education
    Annual Salary
    $60,000.00 - $80,000.00
    SA - Adelaide
    22-Nov-2024
    Apply
  • 3-5 years experience
    Bachelor education
    Annual Salary
    $80,000.00 - $100,000.00
    QLD - Brisbane
    22-Nov-2024
    Apply
You might be interested in Search more similar position >
  • Company
  • VIC
  • 6-10
  • 0-1 Million
Recruitment Consultancy Company
Position Liked
You can view this position inthe Position Liked page.
You are applying Lead Big Data Position,please select the resume you want to post
Successful apply
Congratulations on your successful job application. Please wait patiently for the reply from the enterprise
Positions you may be interested in:
Please not that you saw it in Dimond Pony when you contact me.
Edison CHEN
035****2560
E-mail:
Address: Suite 5, 1/73 Malop Street,Geelong,VIC 3220

Copyright C 2009-2024 Dimond Pony Trading Pty Ltd. All Rights Reserved

Address: Suite 5, 1/73 Malop Street, Geelong VIC 3220 Email: admin@dimondpony.com