CMS GC源码阅读之collect_in_background

概述

collect_in_background由ConcurrentMarkSweepThread线程执行,默认每2秒钟检查一次是否要进行bacgroud gc,如果满足如下条件之一,则进行gc:

  • 请求full gc(_full_gc_requested=true)
  • UseCMSInitiatingOccupancyOnly=false:统计数据有效且完成一次cms gc的时间小于cms generation变满的时间或统计数据无效且cms的使用率大于CMSBootstrapOccupancy(默认50%)
  • 老年代内存使用率大于_initiating_occupancy (默认为92%)
  • UseCMSInitiatingOccupancyOnly=false,且是因为分配对象时内存不足导致的扩容

JVM启动时会计算_initiating_occupancy:

void ConcurrentMarkSweepGeneration::init_initiating_occupancy(intx io, uintx tr) {
  assert(io <= 100 && tr <= 100, "Check the arguments");
  if (io >= 0) {
    _initiating_occupancy = (double)io / 100.0;
  } else {
    _initiating_occupancy = ((100 - MinHeapFreeRatio) +
                             (double)(tr * MinHeapFreeRatio) / 100.0)
                            / 100.0;
  }
}

此处的io为CMSInitiatingOccupancyFraction(默认-1),tr为CMSTriggerRatio(80),MinHeapFreeRatio默认为40,因此默认_initiating_occupancy=92%

CMS GC流程如下:

  1. 如果正在进行foregroud gc或UseAsyncConcMarkSweepGC=false,则放弃执行;
  2. 判断是否需要卸载类:
    • System.gc()并开启ExplicitGCInvokesConcurrentAndUnloadsClasses,则需要卸载类;
    • 开启CMSClassUnloadingEnabled,且自上次卸载类之后发生gc的次数大于CMSClassUnloadingMaxInterval(默认0)或老年代内存使用率大于CMSIsTooFullPercentage(默认98)
      从上面可以看到,默认情况下,一旦开启CMSClassUnloadingEnabled,必然会卸载类
  3. 正式开始执行cms gc,分步执行下列步骤:
  • InitialMarking:
  • Marking
  • Precleaning
  • AbortablePreclean
  • FinalMarking
  • Sweeping
  • Resizing
  • Resetting

InitialMarking

通过vm线程执行VM_CMS_Initial_Mark,由于必须到达安全点,因此需要暂停业务线程,最终调用的逻辑为:

void CMSParInitialMarkTask::work(uint worker_id) {
  elapsedTimer _timer;
  ResourceMark rm;
  HandleMark   hm;

  // ---------- scan from roots --------------
  _timer.start();
  GenCollectedHeap* gch = GenCollectedHeap::heap();
  Par_MarkRefsIntoClosure par_mri_cl(_collector->_span, &(_collector->_markBitMap));
  CMKlassClosure klass_closure(&par_mri_cl);

  // ---------- young gen roots --------------
  {
    work_on_young_gen_roots(worker_id, &par_mri_cl);
    _timer.stop();
    if (PrintCMSStatistics != 0) {
      gclog_or_tty->print_cr(
        "Finished young gen initial mark scan work in %dth thread: %3.3f sec",
        worker_id, _timer.seconds());
    }
  }

  // ---------- remaining roots --------------
  _timer.reset();
  _timer.start();
  gch->gen_process_strong_roots(_collector->_cmsGen->level(),
                                false,     // yg was scanned above
                                false,     // this is parallel code
                                false,     // not scavenging
                                SharedHeap::ScanningOption(_collector->CMSCollector::roots_scanning_options()),
                                &par_mri_cl,
                                true,   // walk all of code cache if (so & SO_CodeCache)
                                NULL,
                                &klass_closure);
  assert(_collector->should_unload_classes()
         || (_collector->CMSCollector::roots_scanning_options() & SharedHeap::SO_CodeCache),
         "if we didn't scan the code cache, we have to be ready to drop nmethods with expired weak oops");
  _timer.stop();
  if (PrintCMSStatistics != 0) {
    gclog_or_tty->print_cr(
      "Finished remaining root initial mark scan work in %dth thread: %3.3f sec",
      worker_id, _timer.seconds());
  }
}

Marking

并发标记在ConcurrentMarkSweepThread线程执行,和业务线程并行,最终的执行逻辑为:

bool CMSCollector::do_marking_st(bool asynch) {
  ResourceMark rm;
  HandleMark   hm;

  // Temporarily make refs discovery single threaded (non-MT)
  ReferenceProcessorMTDiscoveryMutator rp_mut_discovery(ref_processor(), false);
  MarkFromRootsClosure markFromRootsClosure(this, _span, &_markBitMap,
    &_markStack, CMSYield && asynch);
  // the last argument to iterate indicates whether the iteration
  // should be incremental with periodic yields.
  _markBitMap.iterate(&markFromRootsClosure);
  // If _restart_addr is non-NULL, a marking stack overflow
  // occurred; we need to do a fresh iteration from the
  // indicated restart address.
  while (_restart_addr != NULL) {
    if (_foregroundGCIsActive && asynch) {
      // We may be running into repeated stack overflows, having
      // reached the limit of the stack size, while making very
      // slow forward progress. It may be best to bail out and
      // let the foreground collector do its job.
      // Clear _restart_addr, so that foreground GC
      // works from scratch. This avoids the headache of
      // a "rescan" which would otherwise be needed because
      // of the dirty mod union table & card table.
      _restart_addr = NULL;
      return false;  // indicating failure to complete marking
    }
    // Deal with stack overflow:
    // we restart marking from _restart_addr
    HeapWord* ra = _restart_addr;
    markFromRootsClosure.reset(ra);
    _restart_addr = NULL;
    _markBitMap.iterate(&markFromRootsClosure, ra, _span.end());
  }
  return true;
}

Precleaning

预清理在ConcurrentMarkSweepThread线程执行,和业务线程并行,最终的执行逻辑为:

``
## AbortablePreclean
可取消的预清理在ConcurrentMarkSweepThread线程执行,和业务线程并行,最终的执行逻辑为:
```c

FinalMarking

最终标记在vm线程执行,会暂停业务线程,最终执行逻辑为:

if (CMSScavengeBeforeRemark) {
      GenCollectedHeap* gch = GenCollectedHeap::heap();
      // Temporarily set flag to false, GCH->do_collection will
      // expect it to be false and set to true
      FlagSetting fl(gch->_is_gc_active, false);
      NOT_PRODUCT(GCTraceTime t("Scavenge-Before-Remark",
        PrintGCDetails && Verbose, true, _gc_timer_cm);)
      int level = _cmsGen->level() - 1;
      if (level >= 0) {
        gch->do_collection(true,        // full (i.e. force, see below)
                           false,       // !clear_all_soft_refs
                           0,           // size
                           false,       // is_tlab
                           level        // max_level
                          );
      }
    }
    FreelistLocker x(this);
    MutexLockerEx y(bitMapLock(),
                    Mutex::_no_safepoint_check_flag);
    assert(!init_mark_was_synchronous, "but that's impossible!");
    checkpointRootsFinalWork(asynch, clear_all_soft_refs, false);
void CMSCollector::checkpointRootsFinalWork(bool asynch,
  bool clear_all_soft_refs, bool init_mark_was_synchronous) {

  NOT_PRODUCT(GCTraceTime tr("checkpointRootsFinalWork", PrintGCDetails, false, _gc_timer_cm);)

  assert(haveFreelistLocks(), "must have free list locks");
  assert_lock_strong(bitMapLock());

  if (UseAdaptiveSizePolicy) {
    size_policy()->checkpoint_roots_final_begin();
  }

  ResourceMark rm;
  HandleMark   hm;

  GenCollectedHeap* gch = GenCollectedHeap::heap();

  if (should_unload_classes()) {
    CodeCache::gc_prologue();
  }
  assert(haveFreelistLocks(), "must have free list locks");
  assert_lock_strong(bitMapLock());

  if (!init_mark_was_synchronous) {
    // We might assume that we need not fill TLAB's when
    // CMSScavengeBeforeRemark is set, because we may have just done
    // a scavenge which would have filled all TLAB's -- and besides
    // Eden would be empty. This however may not always be the case --
    // for instance although we asked for a scavenge, it may not have
    // happened because of a JNI critical section. We probably need
    // a policy for deciding whether we can in that case wait until
    // the critical section releases and then do the remark following
    // the scavenge, and skip it here. In the absence of that policy,
    // or of an indication of whether the scavenge did indeed occur,
    // we cannot rely on TLAB's having been filled and must do
    // so here just in case a scavenge did not happen.
    gch->ensure_parsability(false);  // fill TLAB's, but no need to retire them
    // Update the saved marks which may affect the root scans.
    gch->save_marks();

    if (CMSPrintEdenSurvivorChunks) {
      print_eden_and_survivor_chunk_arrays();
    }

    {
      COMPILER2_PRESENT(DerivedPointerTableDeactivate dpt_deact;)

      // Note on the role of the mod union table:
      // Since the marker in "markFromRoots" marks concurrently with
      // mutators, it is possible for some reachable objects not to have been
      // scanned. For instance, an only reference to an object A was
      // placed in object B after the marker scanned B. Unless B is rescanned,
      // A would be collected. Such updates to references in marked objects
      // are detected via the mod union table which is the set of all cards
      // dirtied since the first checkpoint in this GC cycle and prior to
      // the most recent young generation GC, minus those cleaned up by the
      // concurrent precleaning.
      if (CMSParallelRemarkEnabled && CollectedHeap::use_parallel_gc_threads()) {
        GCTraceTime t("Rescan (parallel) ", PrintGCDetails, false, _gc_timer_cm);
        do_remark_parallel();
      } else {
        GCTraceTime t("Rescan (non-parallel) ", PrintGCDetails, false,
                    _gc_timer_cm);
        do_remark_non_parallel();
      }
    }
  } else {
    assert(!asynch, "Can't have init_mark_was_synchronous in asynch mode");
    // The initial mark was stop-world, so there's no rescanning to
    // do; go straight on to the next step below.
  }
  verify_work_stacks_empty();
  verify_overflow_empty();

  {
    NOT_PRODUCT(GCTraceTime ts("refProcessingWork", PrintGCDetails, false, _gc_timer_cm);)
    refProcessingWork(asynch, clear_all_soft_refs);
  }
  verify_work_stacks_empty();
  verify_overflow_empty();

  if (should_unload_classes()) {
    CodeCache::gc_epilogue();
  }
  JvmtiExport::gc_epilogue();

  // If we encountered any (marking stack / work queue) overflow
  // events during the current CMS cycle, take appropriate
  // remedial measures, where possible, so as to try and avoid
  // recurrence of that condition.
  assert(_markStack.isEmpty(), "No grey objects");
  size_t ser_ovflw = _ser_pmc_remark_ovflw + _ser_pmc_preclean_ovflw +
                     _ser_kac_ovflw        + _ser_kac_preclean_ovflw;
  if (ser_ovflw > 0) {
    if (PrintCMSStatistics != 0) {
      gclog_or_tty->print_cr("Marking stack overflow (benign) "
        "(pmc_pc="SIZE_FORMAT", pmc_rm="SIZE_FORMAT", kac="SIZE_FORMAT
        ", kac_preclean="SIZE_FORMAT")",
        _ser_pmc_preclean_ovflw, _ser_pmc_remark_ovflw,
        _ser_kac_ovflw, _ser_kac_preclean_ovflw);
    }
    _markStack.expand();
    _ser_pmc_remark_ovflw = 0;
    _ser_pmc_preclean_ovflw = 0;
    _ser_kac_preclean_ovflw = 0;
    _ser_kac_ovflw = 0;
  }
  if (_par_pmc_remark_ovflw > 0 || _par_kac_ovflw > 0) {
    if (PrintCMSStatistics != 0) {
      gclog_or_tty->print_cr("Work queue overflow (benign) "
        "(pmc_rm="SIZE_FORMAT", kac="SIZE_FORMAT")",
        _par_pmc_remark_ovflw, _par_kac_ovflw);
    }
    _par_pmc_remark_ovflw = 0;
    _par_kac_ovflw = 0;
  }
  if (PrintCMSStatistics != 0) {
     if (_markStack._hit_limit > 0) {
       gclog_or_tty->print_cr(" (benign) Hit max stack size limit ("SIZE_FORMAT")",
                              _markStack._hit_limit);
     }
     if (_markStack._failed_double > 0) {
       gclog_or_tty->print_cr(" (benign) Failed stack doubling ("SIZE_FORMAT"),"
                              " current capacity "SIZE_FORMAT,
                              _markStack._failed_double,
                              _markStack.capacity());
     }
  }
  _markStack._hit_limit = 0;
  _markStack._failed_double = 0;

  if ((VerifyAfterGC || VerifyDuringGC) &&
      GenCollectedHeap::heap()->total_collections() >= VerifyGCStartAt) {
    verify_after_remark();
  }

  _gc_tracer_cm->report_object_count_after_gc(&_is_alive_closure);

  // Change under the freelistLocks.
  _collectorState = Sweeping;
  // Call isAllClear() under bitMapLock
  assert(_modUnionTable.isAllClear(),
      "Should be clear by end of the final marking");
  assert(_ct->klass_rem_set()->mod_union_is_clear(),
      "Should be clear by end of the final marking");
  if (UseAdaptiveSizePolicy) {
    size_policy()->checkpoint_roots_final_end(gch->gc_cause());
  }
}

Sweeping

清理在ConcurrentMarkSweepThread线程执行,和业务线程并行,最终的执行逻辑为:

void CMSCollector::sweepWork(ConcurrentMarkSweepGeneration* gen,
  bool asynch) {
  // We iterate over the space(s) underlying this generation,
  // checking the mark bit map to see if the bits corresponding
  // to specific blocks are marked or not. Blocks that are
  // marked are live and are not swept up. All remaining blocks
  // are swept up, with coalescing on-the-fly as we sweep up
  // contiguous free and/or garbage blocks:
  // We need to ensure that the sweeper synchronizes with allocators
  // and stop-the-world collectors. In particular, the following
  // locks are used:
  // . CMS token: if this is held, a stop the world collection cannot occur
  // . freelistLock: if this is held no allocation can occur from this
  //                 generation by another thread
  // . bitMapLock: if this is held, no other thread can access or update
  //

  // Note that we need to hold the freelistLock if we use
  // block iterate below; else the iterator might go awry if
  // a mutator (or promotion) causes block contents to change
  // (for instance if the allocator divvies up a block).
  // If we hold the free list lock, for all practical purposes
  // young generation GC's can't occur (they'll usually need to
  // promote), so we might as well prevent all young generation
  // GC's while we do a sweeping step. For the same reason, we might
  // as well take the bit map lock for the entire duration

  // check that we hold the requisite locks
  assert(have_cms_token(), "Should hold cms token");
  assert(   (asynch && ConcurrentMarkSweepThread::cms_thread_has_cms_token())
         || (!asynch && ConcurrentMarkSweepThread::vm_thread_has_cms_token()),
        "Should possess CMS token to sweep");
  assert_lock_strong(gen->freelistLock());
  assert_lock_strong(bitMapLock());

  assert(!_inter_sweep_timer.is_active(), "Was switched off in an outer context");
  assert(_intra_sweep_timer.is_active(),  "Was switched on  in an outer context");
  gen->cmsSpace()->beginSweepFLCensus((float)(_inter_sweep_timer.seconds()),
                                      _inter_sweep_estimate.padded_average(),
                                      _intra_sweep_estimate.padded_average());
  gen->setNearLargestChunk();

  {
    SweepClosure sweepClosure(this, gen, &_markBitMap,
                            CMSYield && asynch);
    gen->cmsSpace()->blk_iterate_careful(&sweepClosure);
    // We need to free-up/coalesce garbage/blocks from a
    // co-terminal free run. This is done in the SweepClosure
    // destructor; so, do not remove this scope, else the
    // end-of-sweep-census below will be off by a little bit.
  }
  gen->cmsSpace()->sweep_completed();
  gen->cmsSpace()->endSweepFLCensus(sweep_count());
  if (should_unload_classes()) {                // unloaded classes this cycle,
    _concurrent_cycles_since_last_unload = 0;   // ... reset count
  } else {                                      // did not unload classes,
    _concurrent_cycles_since_last_unload++;     // ... increment count
  }
}

Resizing

重新计算容量在ConcurrentMarkSweepThread线程执行,和业务线程并行,最终的执行逻辑为:

void CMSCollector::compute_new_size() {
  assert_locked_or_safepoint(Heap_lock);
  FreelistLocker z(this);
  //重新计算Metaspace的大小,默认情况下,gc之后Metaspace的空闲比例必须大于等于40%否则需要扩容;空闲比例大于70%则需要减小容量
  MetaspaceGC::compute_new_size();
 //重新计算老年代的大小,默认情况下,gc之后老年代的空闲比例必须大于等于40%否则需要扩容;空闲比例大于70%则需要减小容量
  _cmsGen->compute_new_size_free_list();
}

Resetting

重置在ConcurrentMarkSweepThread线程执行,和业务线程并行:

CMS 日志解析

  1. 2018-02-09T09:57:40.697+0800: 40.746: [GC (CMS Initial Mark) [1 CMS-initial-mark: 435454K(1585152K)] 600948K(2972160K), 0.0357643 secs] [Times: user=0.13 sys=0.01, real=0.04 secs]

    • 1表示level,老年代的level为1
    • 435454K为老年代已使用内存,1585152K表示老年代容量
    • 600948K表示堆已使用内存,2972160K表示堆容量
  2. 2018-02-09T09:57:40.733+0800: 40.783: [CMS-concurrent-mark-start]

  3. 2018-02-09T09:57:42.109+0800: 42.159: [CMS-concurrent-mark: 1.155/1.376 secs] [Times: user=4.77 sys=0.24, real=1.37 secs]

    • 1.155表示并行标记耗费时间,1.376表示并行标记开始到结束的系统时间
  4. 2018-02-09T09:57:42.109+0800: 42.159: [CMS-concurrent-preclean-start]

  5. 2018-02-09T09:57:42.119+0800: 42.169: [CMS-concurrent-preclean: 0.010/0.010 secs] [Times: user=0.03 sys=0.00, real=0.01 secs]

  6. 2018-02-09T09:57:42.120+0800: 42.169: [CMS-concurrent-abortable-preclean-start]
    CMS: abort preclean due to time

  7. 2018-02-09T09:57:47.419+0800: 47.469: [CMS-concurrent-abortable-preclean: 5.024/5.299 secs] [Times: user=8.06 sys=0.16, real=5.30 secs]

  8. 2018-02-09T09:57:47.420+0800: 47.470: [GC (CMS Final Remark) [YG occupancy: 575396 K (1387008 K)]

  9. 2018-02-09T09:57:47.420+0800: 47.470: [Rescan (parallel) , 0.1041595 secs]

  10. 2018-02-09T09:57:47.524+0800: 47.574: [weak refs processing, 0.0000340 secs]

  11. 2018-02-09T09:57:47.525+0800: 47.574: [class unloading, 0.0216172 secs]

  12. 2018-02-09T09:57:47.546+0800: 47.596: [scrub symbol table, 0.0288155 secs]

  13. 2018-02-09T09:57:47.575+0800: 47.624: [scrub string table, 0.0024586 secs][1 CMS-remark: 435454K(1585152K)] 1010850K(2972160K), 0.1618950 secs] [Times: user=0.47 sys=0.00, real=0.17 secs]

  14. 2018-02-09T09:57:47.582+0800: 47.632: [CMS-concurrent-sweep-start]

  15. 2018-02-09T09:57:47.843+0800: 47.892: [CMS-concurrent-sweep: 0.260/0.260 secs] [Times: user=0.28 sys=0.00, real=0.26 secs]

  16. 2018-02-09T09:57:47.843+0800: 47.892: [CMS-concurrent-reset-start]

  17. 2018-02-09T09:57:47.846+0800: 47.895: [CMS-concurrent-reset: 0.003/0.003 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]