python(08)实践Django-异步任务celery

本项目使用celery 异步去同步项目在jenkins中的构建信息。

最终结果展示:

  1. 在项目的构建列表中点击同步按钮,调用celery去jenkins同步所有构建信息


    Paste_Image.png
  2. 每隔12小时,调用celery的定时任务去jenkins同步最新build信息


    Paste_Image.png

前提:已有redis服务并启动

  1. 安装celery
      pip install celery
      pip install redis
    
  2. settings.py 配置
     BROKER_URL = 'redis://ip:6379/6'
     CELERY_RESULT_BACKEND = BROKER_URL
     CELERY_TIMEZONE = 'Asia/Shanghai'
     #时区不一致时会导致任务监控不到
    
  3. init.py
    from .celery import app as celery_app
    
  4. celery.py 配置
     # ~*~ coding: utf-8 ~*~
     from __future__ import absolute_import, unicode_literals
     import os
     from celery import Celery
     from celery.schedules import crontab
     from django.conf import settings
    
     # set the default Django settings module for the 'celery' program.
     os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'Testmanager.settings')
     app = Celery('Testmanager')
    
     # Using a string here means the worker will not have to
     # pickle the object when using Windows.
     app.config_from_object('django.conf:settings')
     app.autodiscover_tasks(lambda: [app_config.split('.')[0]
                                 for app_config in settings.INSTALLED_APPS])
      #定时任务配置
      app.conf.update(
     CELERYBEAT_SCHEDULE={
        'refresh-pci-last-build-info': {
             'task': 'devops.tasks.update_all_pcis_by_jenkins',
            'schedule': crontab(hour='*/12'),
             'args': (),
             }
         }
     )
    
  5. task任务
    @shared_task
    def update_all_pcis_by_jenkins():
     jen = jenkins.Jenkins(jenkins_url, username=jenkins_user, password=jenkins_passwd)
     pcis = PCI.objects.all()
     for pci in pcis :
         job_name = pci.job_name
         lastBuild_Number = jen.get_job_info(job_name)['lastBuild']['number']
         if pci.last_build_number != lastBuild_Number :
             lastBuild_url = jen.get_build_info(job_name, lastBuild_Number)['url']  # 最后一次build号
             times = jen.get_build_info(job_name, lastBuild_Number)['timestamp']  # build的日期
             result = jen.get_build_info(job_name, lastBuild_Number)['result']
             pci.last_build_number = lastBuild_Number
             pci.last_build_result = result
             pci.last_build_url = lastBuild_url
             pci.last_update_time = timestamp2datetime(times)
             pci.save()
         print(job_name)
         print('已是最新版本')
    @shared_task
    def update_pcis_by_jenkins(job_name,pid):
      jen = jenkins.Jenkins(jenkins_url, username=jenkins_user, password=jenkins_passwd)
      print (job_name)
      pci = PCI.objects.get(id=pid)
      job_info_scm = jen.get_job_info(job_name)['scm']['_class']
     #{'_class': 'hudson.plugins.git.GitSCM'}
      print('job_info_scm')
      print(job_info_scm)
      if job_info_scm.find('git') != -1:
          pci.pro_scm = 'git'
     else:
         pci.pro_scm = 'svn'
      lastBuild_Number = jen.get_job_info(job_name)['lastBuild']['number']
      lastBuild_url = jen.get_build_info(job_name, lastBuild_Number)['url']# 最后一次build号
      times = jen.get_build_info(job_name, lastBuild_Number)['timestamp']# build的日期
      result = jen.get_build_info(job_name, lastBuild_Number)['result']
      pci.last_build_number = lastBuild_Number
      pci.last_build_result = result
      pci.last_build_url = lastBuild_url
      pci.last_update_time = timestamp2datetime(times)
      pci.save()
    
  6. 启动celery 进程
    celery -A Testmanager worker --loglevel=info
    #定时任务
    celery -A Testmanager beat --loglevel=info
    
加载已有的tasks

定时任务

推荐阅读更多精彩内容

  • 在工作中遇到的开发对logging的使用是分两个极端的,有些人喜欢把什么信息都打印出来,一不小心日志就刷屏,另一些...
    灼灼2015阅读 423评论 0 1
  • Spring Cloud为开发人员提供了快速构建分布式系统中一些常见模式的工具(例如配置管理,服务发现,断路器,智...
    卡卡罗2017阅读 121,074评论 16 134
  • 目的:通过访问Jenkins得到各个项目的构建状态。 jenkins是一款持续集成的工具,插件众多,配置简单,跨平...
    灼灼2015阅读 1,571评论 0 2
  • =========================================================...
    lavor阅读 2,748评论 0 5
  • 去电影院看了《滚蛋吧,肿瘤君》,果然还是哭了,泪流满面,狼狈不堪。 小伙伴为了给熊顿打气一起剃了光头齐齐出现在她面...
    季夏习习阅读 406评论 2 3