/* -*- Mode: C++; tab-width: 8; indent-tabs-mode: nil; c-basic-offset: 2 -*- * vim: set ts=8 sts=2 et sw=2 tw=80: * This Source Code Form is subject to the terms of the Mozilla Public * License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */
/* * [SMDOC] Garbage Collector * * This code implements an incremental mark-and-sweep garbage collector, with * most sweeping carried out in the background on a parallel thread. * * Full vs. zone GC * ---------------- * * The collector can collect all zones at once, or a subset. These types of * collection are referred to as a full GC and a zone GC respectively. * * It is possible for an incremental collection that started out as a full GC to * become a zone GC if new zones are created during the course of the * collection. * * Incremental collection * ---------------------- * * For a collection to be carried out incrementally the following conditions * must be met: * - the collection must be run by calling js::GCSlice() rather than js::GC() * - the GC parameter JSGC_INCREMENTAL_GC_ENABLED must be true. * * The last condition is an engine-internal mechanism to ensure that incremental * collection is not carried out without the correct barriers being implemented. * For more information see 'Incremental marking' below. * * If the collection is not incremental, all foreground activity happens inside * a single call to GC() or GCSlice(). However the collection is not complete * until the background sweeping activity has finished. * * An incremental collection proceeds as a series of slices, interleaved with * mutator activity, i.e. running JavaScript code. Slices are limited by a time * budget. The slice finishes as soon as possible after the requested time has * passed. * * Collector states * ---------------- * * The collector proceeds through the following states, the current state being * held in JSRuntime::gcIncrementalState: * * - Prepare - unmarks GC things, discards JIT code and other setup * - MarkRoots - marks the stack and other roots * - Mark - incrementally marks reachable things * - Sweep - sweeps zones in groups and continues marking unswept zones * - Finalize - performs background finalization, concurrent with mutator * - Compact - incrementally compacts by zone * - Decommit - performs background decommit and chunk removal * * Roots are marked in the first MarkRoots slice; this is the start of the GC * proper. The following states can take place over one or more slices. * * In other words an incremental collection proceeds like this: * * Slice 1: Prepare: Starts background task to unmark GC things * * ... JS code runs, background unmarking finishes ... * * Slice 2: MarkRoots: Roots are pushed onto the mark stack. * Mark: The mark stack is processed by popping an element, * marking it, and pushing its children. * * ... JS code runs ... * * Slice 3: Mark: More mark stack processing. * * ... JS code runs ... * * Slice n-1: Mark: More mark stack processing. * * ... JS code runs ... * * Slice n: Mark: Mark stack is completely drained. * Sweep: Select first group of zones to sweep and sweep them. * * ... JS code runs ... * * Slice n+1: Sweep: Mark objects in unswept zones that were newly * identified as alive (see below). Then sweep more zone * sweep groups. * * ... JS code runs ... * * Slice n+2: Sweep: Mark objects in unswept zones that were newly * identified as alive. Then sweep more zones. * * ... JS code runs ... * * Slice m: Sweep: Sweeping is finished, and background sweeping * started on the helper thread. * * ... JS code runs, remaining sweeping done on background thread ... * * When background sweeping finishes the GC is complete. * * Incremental marking * ------------------- * * Incremental collection requires close collaboration with the mutator (i.e., * JS code) to guarantee correctness. * * - During an incremental GC, if a memory location (except a root) is written * to, then the value it previously held must be marked. Write barriers * ensure this. * * - Any object that is allocated during incremental GC must start out marked. * * - Roots are marked in the first slice and hence don't need write barriers. * Roots are things like the C stack and the VM stack. * * The problem that write barriers solve is that between slices the mutator can * change the object graph. We must ensure that it cannot do this in such a way * that makes us fail to mark a reachable object (marking an unreachable object * is tolerable). * * We use a snapshot-at-the-beginning algorithm to do this. This means that we * promise to mark at least everything that is reachable at the beginning of * collection. To implement it we mark the old contents of every non-root memory * location written to by the mutator while the collection is in progress, using * write barriers. This is described in gc/Barrier.h. * * Incremental sweeping * -------------------- * * Sweeping is difficult to do incrementally because object finalizers must be * run at the start of sweeping, before any mutator code runs. The reason is * that some objects use their finalizers to remove themselves from caches. If * mutator code was allowed to run after the start of sweeping, it could observe * the state of the cache and create a new reference to an object that was just * about to be destroyed. * * Sweeping all finalizable objects in one go would introduce long pauses, so * instead sweeping broken up into groups of zones. Zones which are not yet * being swept are still marked, so the issue above does not apply. * * The order of sweeping is restricted by cross compartment pointers - for * example say that object |a| from zone A points to object |b| in zone B and * neither object was marked when we transitioned to the Sweep phase. Imagine we * sweep B first and then return to the mutator. It's possible that the mutator * could cause |a| to become alive through a read barrier (perhaps it was a * shape that was accessed via a shape table). Then we would need to mark |b|, * which |a| points to, but |b| has already been swept. * * So if there is such a pointer then marking of zone B must not finish before * marking of zone A. Pointers which form a cycle between zones therefore * restrict those zones to being swept at the same time, and these are found * using Tarjan's algorithm for finding the strongly connected components of a * graph. * * GC things without finalizers, and things with finalizers that are able to run * in the background, are swept on the background thread. This accounts for most * of the sweeping work. * * Reset * ----- * * During incremental collection it is possible, although unlikely, for * conditions to change such that incremental collection is no longer safe. In * this case, the collection is 'reset' by resetIncrementalGC(). If we are in * the mark state, this just stops marking, but if we have started sweeping * already, we continue non-incrementally until we have swept the current sweep * group. Following a reset, a new collection is started. * * Compacting GC * ------------- * * Compacting GC happens at the end of a major GC as part of the last slice. * There are three parts: * * - Arenas are selected for compaction. * - The contents of those arenas are moved to new arenas. * - All references to moved things are updated. * * Collecting Atoms * ---------------- * * Atoms are collected differently from other GC things. They are contained in * a special zone and things in other zones may have pointers to them that are * not recorded in the cross compartment pointer map. Each zone holds a bitmap * with the atoms it might be keeping alive, and atoms are only collected if * they are not included in any zone's atom bitmap. See AtomMarking.cpp for how * this bitmap is managed.
*/
using mozilla::EnumSet; using mozilla::MakeScopeExit; using mozilla::Maybe; using mozilla::Nothing; using mozilla::Some; using mozilla::TimeDuration; using mozilla::TimeStamp;
using JS::SliceBudget; using JS::TimeBudget; using JS::WorkBudget;
static_assert(std::size(slotsToThingKind) == SLOTS_TO_THING_KIND_LIMIT, "We have defined a slot count for each kind.");
// A table converting an object size in "slots" (increments of // sizeof(js::Value)) to the total number of bytes in the corresponding // AllocKind. See gc::slotsToThingKind. This primarily allows wasm jit code to // remain compliant with the AllocKind system. // // To use this table, subtract sizeof(NativeObject) from your desired allocation // size, divide by sizeof(js::Value) to get the number of "slots", and then // index into this table. See gc::GetGCObjectKindForBytes. const constexpr uint32_t gc::slotsToAllocKindBytes[] = { // These entries correspond exactly to gc::slotsToThingKind. The numeric // comments therefore indicate the number of slots that the "bytes" would // correspond to. // clang-format off /* 0 */ sizeof(JSObject_Slots0), sizeof(JSObject_Slots2), sizeof(JSObject_Slots2), sizeof(JSObject_Slots4), /* 4 */ sizeof(JSObject_Slots4), sizeof(JSObject_Slots8), sizeof(JSObject_Slots8), sizeof(JSObject_Slots8), /* 8 */ sizeof(JSObject_Slots8), sizeof(JSObject_Slots12), sizeof(JSObject_Slots12), sizeof(JSObject_Slots12), /* 12 */ sizeof(JSObject_Slots12), sizeof(JSObject_Slots16), sizeof(JSObject_Slots16), sizeof(JSObject_Slots16), /* 16 */ sizeof(JSObject_Slots16) // clang-format on
};
// Please also update jit-test/tests/gc/gczeal.js when updating this help text. // clang-format off constchar gc::ZealModeHelpText[] = " Specifies how zealous the garbage collector should be. Some of these modes\n" " can be set simultaneously, by passing multiple level options, e.g. \"2;4\"\n" " will activate both modes 2 and 4. Modes can be specified by name or\n" " number.\n" " \n" " Values:\n" " 0: (None) Normal amount of collection (resets all modes)\n" " 1: (RootsChange) Collect when roots are added or removed\n" " 2: (Alloc) Collect when every N allocations (default: 100)\n" " 4: (VerifierPre) Verify pre write barriers between instructions\n" " 6: (YieldBeforeRootMarking) Incremental GC in two slices that yields\n" " before root marking\n" " 7: (GenerationalGC) Collect the nursery every N nursery allocations\n" " 8: (YieldBeforeMarking) Incremental GC in two slices that yields\n" " between the root marking and marking phases\n" " 9: (YieldBeforeSweeping) Incremental GC in two slices that yields\n" " between the marking and sweeping phases\n" " 10: (IncrementalMultipleSlices) Incremental GC in many slices\n" " 11: (IncrementalMarkingValidator) Verify incremental marking\n" " 12: (ElementsBarrier) Use the individual element post-write barrier\n" " regardless of elements size\n" " 13: (CheckHashTablesOnMinorGC) Check internal hashtables on minor GC\n" " 14: (Compact) Perform a shrinking collection every N allocations\n" " 15: (CheckHeapAfterGC) Walk the heap to check its integrity after every\n" " GC\n" " 17: (YieldBeforeSweepingAtoms) Incremental GC in two slices that yields\n" " before sweeping the atoms table\n" " 18: (CheckGrayMarking) Check gray marking invariants after every GC\n" " 19: (YieldBeforeSweepingCaches) Incremental GC in two slices that yields\n" " before sweeping weak caches\n" " 21: (YieldBeforeSweepingObjects) Incremental GC that yields once per\n" " zone before sweeping foreground finalized objects\n" " 22: (YieldBeforeSweepingNonObjects) Incremental GC that yields once per\n" " zone before sweeping non-object GC things\n" " 23: (YieldBeforeSweepingPropMapTrees) Incremental GC that yields once\n" " per zone before sweeping shape trees\n" " 24: (CheckWeakMapMarking) Check weak map marking invariants after every\n" " GC\n" " 25: (YieldWhileGrayMarking) Incremental GC in two slices that yields\n" " during gray marking\n"; // clang-format on
// The set of zeal modes that yield at specific points in collection. static constexpr EnumSet<ZealMode> YieldPointZealModes = {
ZealMode::YieldBeforeRootMarking,
ZealMode::YieldBeforeMarking,
ZealMode::YieldBeforeSweeping,
ZealMode::YieldBeforeSweepingAtoms,
ZealMode::YieldBeforeSweepingCaches,
ZealMode::YieldBeforeSweepingObjects,
ZealMode::YieldBeforeSweepingNonObjects,
ZealMode::YieldBeforeSweepingPropMapTrees,
ZealMode::YieldWhileGrayMarking};
// The set of zeal modes that control incremental slices. static constexpr EnumSet<ZealMode> IncrementalSliceZealModes =
YieldPointZealModes +
EnumSet<ZealMode>{ZealMode::IncrementalMultipleSlices};
// The set of zeal modes that trigger GC periodically. static constexpr EnumSet<ZealMode> PeriodicGCZealModes =
IncrementalSliceZealModes + EnumSet<ZealMode>{ZealMode::Alloc,
ZealMode::GenerationalGC,
ZealMode::Compact};
// The set of zeal modes that are mutually exclusive. All of these trigger GC // except VerifierPre. static constexpr EnumSet<ZealMode> ExclusiveZealModes =
PeriodicGCZealModes + EnumSet<ZealMode>{ZealMode::VerifierPre};
// Modes that trigger periodically are mutually exclusive. If we're setting // one of those, we first reset all of them.
ZealMode zealMode = ZealMode(zeal); if (ExclusiveZealModes.contains(zealMode)) { for (auto mode : ExclusiveZealModes) { if (hasZealMode(mode)) {
clearZealMode(mode);
}
}
}
if (zealMode == ZealMode::GenerationalGC) {
evictNursery(JS::GCReason::EVICT_NURSERY);
nursery().enterZealMode();
}
bool GCRuntime::parseAndSetZeal(constchar* str) { // Set the zeal mode from a string consisting of one or more mode specifiers // separated by ';', optionally followed by a ',' and the trigger frequency. // The mode specifiers can by a mode name or its number.
auto text = CharRange(str, strlen(str));
CharRangeVector parts; if (!SplitStringBy(text, ',', &parts)) { returnfalse;
}
bool GCRuntime::zealModeControlsYieldPoint() const { // Indicates whether a zeal mode is enabled that controls the point at which // the collector yields to the mutator. Yield can happen once per collection // or once per zone depending on the mode. return hasAnyZealModeOf(YieldPointZealModes);
}
bool GCRuntime::hasZealMode(ZealMode mode) const {
static_assert(size_t(ZealMode::Limit) < sizeof(zealModeBits) * 8, "Zeal modes must fit in zealModeBits"); return zealModeBits & (1 << uint32_t(mode));
}
for (auto& marker : markers) { if (!marker->init()) { returnfalse;
}
}
if (!initSweepActions()) { returnfalse;
}
UniquePtr<Zone> zone = MakeUnique<Zone>(rt, Zone::AtomsZone); if (!zone || !zone->init()) { returnfalse;
}
// The atoms zone is stored as the first element of the zones vector.
MOZ_ASSERT(zone->isAtomsZone());
MOZ_ASSERT(zones().empty());
MOZ_ALWAYS_TRUE(zones().reserve(1)); // ZonesVector has inline capacity 4.
zones().infallibleAppend(zone.release());
// Wait for nursery background free to end and disable it to release memory. if (nursery().isEnabled()) {
nursery().disable();
}
// Wait until the background finalization and allocation stops and the // helper thread shuts down before we forcefully release any remaining GC // memory.
sweepTask.join();
markTask.join();
freeTask.join();
allocTask.cancelAndWait();
decommitTask.cancelAndWait(); #ifdef DEBUG
{
MOZ_ASSERT(dispatchedParallelTasks == 0);
AutoLockHelperThreadState lock;
MOZ_ASSERT(queuedParallelTasks.ref().isEmpty(lock));
} #endif
bool GCRuntime::freezeSharedAtomsZone() { // This is called just after permanent atoms and well-known symbols have been // created. At this point all existing atoms and symbols are permanent. // // This method makes the current atoms zone into a shared atoms zone and // removes it from the zones list. Everything in it is marked black. A new // empty atoms zone is created, where all atoms local to this runtime will // live. // // The shared atoms zone will not be collected until shutdown when it is // returned to the zone list by restoreSharedAtomsZone().
void GCRuntime::restoreSharedAtomsZone() { // Return the shared atoms zone to the zone list. This allows the contents of // the shared atoms zone to be collected when the parent runtime is shut down.
// Insert at start to preserve invariant that atoms zones come first.
AutoEnterOOMUnsafeRegion oomUnsafe; if (!zones().insert(zones().begin(), sharedAtomsZone_)) {
oomUnsafe.crash("restoreSharedAtomsZone");
}
// Special case: if there is still an `AutoDisableGenerationalGC` active (eg // from the --no-ggc command-line flag), then do not allow controlling the // state of the nursery. Done here where cx is available. if (key == JSGC_NURSERY_ENABLED && cx->generationalDisabled > 0) { returnfalse;
}
bool GCRuntime::setParameter(JSGCParamKey key, uint32_t value,
AutoLockGC& lock) { switch (key) { case JSGC_SLICE_TIME_BUDGET_MS:
defaultTimeBudgetMS_ = value; break; case JSGC_INCREMENTAL_GC_ENABLED:
setIncrementalGCEnabled(value != 0); break; case JSGC_PER_ZONE_GC_ENABLED:
perZoneGCEnabled = value != 0; break; case JSGC_COMPACTING_ENABLED:
compactingEnabled = value != 0; break; case JSGC_NURSERY_ENABLED: {
AutoUnlockGC unlock(lock);
setNurseryEnabled(value != 0); break;
} case JSGC_PARALLEL_MARKING_ENABLED:
setParallelMarkingEnabled(value != 0); break; case JSGC_INCREMENTAL_WEAKMAP_ENABLED: for (auto& marker : markers) {
marker->incrementalWeakMapMarkingEnabled = value != 0;
} break; case JSGC_SEMISPACE_NURSERY_ENABLED: {
AutoUnlockGC unlock(lock);
nursery().setSemispaceEnabled(value); break;
} case JSGC_MIN_EMPTY_CHUNK_COUNT:
setMinEmptyChunkCount(value, lock); break; default: if (IsGCThreadParameter(key)) { return setThreadParameter(key, value, lock);
}
if (!tunables.setParameter(key, value)) { returnfalse;
}
updateAllGCStartThresholds();
}
returntrue;
}
bool GCRuntime::setThreadParameter(JSGCParamKey key, uint32_t value,
AutoLockGC& lock) { if (rt->parentRuntime) { // Don't allow these to be set for worker runtimes. returnfalse;
}
switch (key) { case JSGC_HELPER_THREAD_RATIO: if (value == 0) { returnfalse;
}
helperThreadRatio = double(value) / 100.0; break; case JSGC_MAX_HELPER_THREADS: if (value == 0) { returnfalse;
}
maxHelperThreads = value; break; case JSGC_MAX_MARKING_THREADS:
maxMarkingThreads = std::min(size_t(value), MaxParallelWorkers); break; default:
MOZ_CRASH("Unexpected parameter key");
}
void GCRuntime::setNurseryEnabled(bool enabled) { if (enabled) {
nursery().enable();
} else { if (nursery().isEnabled()) {
minorGC(JS::GCReason::EVICT_NURSERY);
nursery().disable();
}
}
}
void GCRuntime::updateHelperThreadCount() { if (!CanUseExtraThreads()) { // startTask will run the work on the main thread if the count is 1.
MOZ_ASSERT(helperThreadCount == 1);
markingThreadCount = 1;
// Number of extra threads required during parallel marking to ensure we can // start the necessary marking tasks. Background free and background // allocation may already be running and we want to avoid these tasks blocking // marking. In real configurations there will be enough threads that this // won't affect anything. static constexpr size_t SpareThreadsDuringParallelMarking = 2;
// Calculate the target thread count for parallel marking, which uses separate // parameters to let us adjust this independently.
markingThreadCount = std::min(cpuCount / 2, maxMarkingThreads.ref());
// Calculate the overall target thread count taking into account the separate // target for parallel marking threads. Add spare threads to avoid blocking // parallel marking when there is other GC work happening.
size_t targetCount =
std::max(helperThreadCount.ref(),
markingThreadCount.ref() + SpareThreadsDuringParallelMarking);
// Attempt to create extra threads if possible. This is not supported when // using an external thread pool.
AutoLockHelperThreadState lock;
(void)HelperThreadState().ensureThreadCount(targetCount, lock);
// Limit all thread counts based on the number of threads available, which may // be fewer than requested.
size_t availableThreadCount = GetHelperThreadCount();
MOZ_ASSERT(availableThreadCount != 0);
targetCount = std::min(targetCount, availableThreadCount);
helperThreadCount = std::min(helperThreadCount.ref(), availableThreadCount); if (availableThreadCount < SpareThreadsDuringParallelMarking) {
markingThreadCount = 1;
} else {
markingThreadCount =
std::min(markingThreadCount.ref(),
availableThreadCount - SpareThreadsDuringParallelMarking);
}
// Update the maximum number of threads that will be used for GC work.
maxParallelThreads = targetCount;
}
bool GCRuntime::initOrDisableParallelMarking() { // Attempt to initialize parallel marking state or disable it on failure. This // is called when parallel marking is enabled or disabled.
MOZ_ASSERT(markers.length() != 0);
if (updateMarkersVector()) { returntrue;
}
// Failed to initialize parallel marking so disable it instead.
MOZ_ASSERT(parallelMarkingEnabled);
parallelMarkingEnabled = false;
MOZ_ALWAYS_TRUE(updateMarkersVector()); returnfalse;
}
// Update the helper thread system's global count by subtracting this // runtime's current contribution |reservedMarkingThreads| and adding the new // contribution |newCount|.
AutoLockHelperThreadState lock; auto& globalCount = HelperThreadState().gcParallelMarkingThreads;
MOZ_ASSERT(globalCount >= reservedMarkingThreads);
size_t newGlobalCount = globalCount - reservedMarkingThreads + newCount; if (newGlobalCount > HelperThreadState().threadCount) { // Not enough total threads. returnfalse;
}
bool GCRuntime::updateMarkersVector() {
MOZ_ASSERT(helperThreadCount >= 1, "There must always be at least one mark task");
MOZ_ASSERT(CurrentThreadCanAccessRuntime(rt));
assertNoMarkingWork();
// Limit worker count to number of GC parallel tasks that can run // concurrently, otherwise one thread can deadlock waiting on another.
size_t targetCount = std::min(markingWorkerCount(), getMaxParallelThreads());
if (rt->isMainRuntime()) { // For the main runtime, reserve helper threads as long as parallel marking // is enabled. Worker runtimes may not mark in parallel if there are // insufficient threads available at the time.
size_t threadsToReserve = targetCount > 1 ? targetCount : 0; if (!reserveMarkingThreads(threadsToReserve)) { returnfalse;
}
}
if (markers.length() > targetCount) { return markers.resize(targetCount);
}
while (markers.length() < targetCount) { auto marker = MakeUnique<GCMarker>(rt); if (!marker) { returnfalse;
}
#ifdef JS_GC_ZEAL if (maybeMarkStackLimit) {
marker->setMaxCapacity(maybeMarkStackLimit);
} #endif
if (!marker->init()) { returnfalse;
}
if (!markers.emplaceBack(std::move(marker))) { returnfalse;
}
}
returntrue;
}
template <typename F> staticbool EraseCallback(CallbackVector<F>& vector, F callback) { for (Callback<F>* p = vector.begin(); p != vector.end(); p++) { if (p->op == callback) {
vector.erase(p); returntrue;
}
}
returnfalse;
}
template <typename F> staticbool EraseCallback(CallbackVector<F>& vector, F callback, void* data) { for (Callback<F>* p = vector.begin(); p != vector.end(); p++) { if (p->op == callback && p->data == data) {
vector.erase(p); returntrue;
}
}
void GCRuntime::removeBlackRootsTracer(JSTraceDataOp traceOp, void* data) { // Can be called from finalizers
MOZ_ALWAYS_TRUE(EraseCallback(blackRootTracers.ref(), traceOp));
}
bool GCRuntime::addRoot(Value* vp, constchar* name) { /* * Sometimes Firefox will hold weak references to objects and then convert * them to strong references by calling AddRoot (e.g., via PreserveWrapper, * or ModifyBusyCount in workers). We need a read barrier to cover these * cases.
*/
MOZ_ASSERT(vp);
Value value = *vp; if (value.isGCThing()) {
ValuePreWriteBarrier(value);
}
bool js::gc::IsCurrentlyAnimating(const TimeStamp& lastAnimationTime, const TimeStamp& currentTime) { // Assume that we're currently animating if js::NotifyAnimationActivity has // been called in the last second. staticconstauto oneSecond = TimeDuration::FromSeconds(1); return !lastAnimationTime.IsNull() &&
currentTime < (lastAnimationTime + oneSecond);
}
bool GCRuntime::shouldCompact() { // Compact on shrinking GC if enabled. Skip compacting in incremental GCs // if we are currently animating, unless the user is inactive or we're // responding to memory pressure.
if (!isShrinkingGC() || !isCompactingGCEnabled()) { returnfalse;
}
bool GCRuntime::triggerGC(JS::GCReason reason) { /* * Don't trigger GCs if this is being called off the main thread from * onTooMuchMalloc().
*/ if (!CurrentThreadCanAccessRuntime(rt)) { returnfalse;
}
/* GC is already running. */ if (JS::RuntimeHeapIsCollecting()) { returnfalse;
}
if (trigger.shouldTrigger) { // Start or continue an in progress incremental GC. We do this to try to // avoid performing non-incremental GCs on zones which allocate a lot of // data, even when incremental slices can't be triggered via scheduling in // the event loop.
triggerZoneGC(zone, JS::GCReason::ALLOC_TRIGGER, trigger.usedBytes,
trigger.thresholdBytes);
}
}
// Trigger a zone GC. budgetIncrementalGC() will work out whether to do an // incremental or non-incremental collection.
triggerZoneGC(zone, reason, trigger.usedBytes, trigger.thresholdBytes); returntrue;
}
bool GCRuntime::shouldDecommit() const { switch (gcOptions()) { case JS::GCOptions::Normal: // If we are allocating heavily enough to trigger "high frequency" GC then // skip decommit so that we do not compete with the mutator. return !schedulingState.inHighFrequencyGCMode(); case JS::GCOptions::Shrink: // If we're doing a shrinking GC we always decommit to release as much // memory as possible. returntrue; case JS::GCOptions::Shutdown: // There's no point decommitting as we are about to free everything. returnfalse;
}
// Verify that all entries in the empty chunks pool are unused. for (ChunkPool::Iter chunk(emptyChunks(lock)); !chunk.done();
chunk.next()) {
MOZ_ASSERT(chunk->unused());
}
} #endif
if (!shouldDecommit()) { return;
}
{
AutoLockGC lock(this); if (availableChunks(lock).empty() && !tooManyEmptyChunks(lock) &&
emptyChunks(lock).empty()) { return; // Nothing to do.
}
}
// To help minimize the total number of chunks needed over time, sort the // available chunks list so that we allocate into more-used chunks first.
gc->availableChunks(gcLock).sort();
if (DecommitEnabled()) {
gc->decommitEmptyChunks(cancel_, gcLock);
gc->decommitFreeArenas(cancel_, gcLock);
}
}
}
// Called from a background thread to decommit free arenas. Releases the GC // lock. void GCRuntime::decommitEmptyChunks(constbool& cancel, AutoLockGC& lock) {
Vector<ArenaChunk*, 0, SystemAllocPolicy> chunksToDecommit; for (ChunkPool::Iter chunk(emptyChunks(lock)); !chunk.done(); chunk.next()) { if (CanDecommitWholeChunk(chunk) && !chunksToDecommit.append(chunk)) {
onOutOfMallocMemory(lock); return;
}
}
for (ArenaChunk* chunk : chunksToDecommit) { if (cancel) { break;
}
// Check whether something used the chunk while lock was released. if (!CanDecommitWholeChunk(chunk)) { continue;
}
// Temporarily remove the chunk while decommitting its memory so that the // mutator doesn't start allocating from it when we drop the lock.
emptyChunks(lock).remove(chunk);
// Called from a background thread to decommit free arenas. Releases the GC // lock. void GCRuntime::decommitFreeArenas(constbool& cancel, AutoLockGC& lock) {
MOZ_ASSERT(DecommitEnabled());
// Since we release the GC lock while doing the decommit syscall below, // it is dangerous to iterate the available list directly, as the active // thread could modify it concurrently. Instead, we build and pass an // explicit Vector containing the Chunks we want to visit.
Vector<ArenaChunk*, 0, SystemAllocPolicy> chunksToDecommit; for (ChunkPool::Iter chunk(availableChunks(lock)); !chunk.done();
chunk.next()) { if (chunk->info.numArenasFreeCommitted != 0 &&
!chunksToDecommit.append(chunk)) {
onOutOfMallocMemory(lock); return;
}
}
for (ArenaChunk* chunk : chunksToDecommit) {
MOZ_ASSERT(chunk->getKind() == ChunkKind::TenuredArenas);
MOZ_ASSERT(!chunk->unused()); if (!chunk->hasAvailableArenas()) { // Chunk has become full while lock was released. continue;
}
// Do all possible decommit immediately from the current thread without // releasing the GC lock or allocating any memory. void GCRuntime::decommitFreeArenasWithoutUnlocking(const AutoLockGC& lock) {
MOZ_ASSERT(DecommitEnabled()); for (ChunkPool::Iter chunk(availableChunks(lock)); !chunk.done();
chunk.next()) {
chunk->decommitFreeArenasWithoutUnlocking(lock);
}
MOZ_ASSERT(availableChunks(lock).verify());
}
void GCRuntime::maybeRequestGCAfterBackgroundTask( const AutoLockHelperThreadState& lock) { if (requestSliceAfterBackgroundTask) { // Request a slice. The main thread may continue the collection immediately // or it may yield to let the embedding schedule a slice.
requestSliceAfterBackgroundTask = false;
requestMajorGC(JS::GCReason::BG_TASK_FINISHED);
}
}
if (!buffersToFreeAfterMinorGC.ref().empty() ||
!stringBuffersToReleaseAfterMinorGC.ref().empty() ||
!largeBuffersToFreeAfterMinorGC.ref().isEmpty()) { // In the rare case that this hasn't processed the buffers from a previous // minor GC we have to wait here.
MOZ_ASSERT(!freeTask.isIdle(lock));
freeTask.joinWithLockHeld(lock);
}
void Realm::destroy(JS::GCContext* gcx) {
JSRuntime* rt = gcx->runtime(); if (auto callback = rt->destroyRealmCallback) {
callback(gcx, this);
} if (principals()) {
JS_DropPrincipals(rt->mainContextFromOwnThread(), principals());
} // Bug 1560019: Malloc memory associated with a zone but not with a specific // GC thing is not currently tracked.
gcx->deleteUntracked(this);
}
void Compartment::destroy(JS::GCContext* gcx) {
JSRuntime* rt = gcx->runtime(); if (auto callback = rt->destroyCompartmentCallback) {
callback(gcx, this);
} // Bug 1560019: Malloc memory associated with a zone but not with a specific // GC thing is not currently tracked.
gcx->deleteUntracked(this);
rt->gc.stats().sweptCompartment();
}
void Zone::destroy(JS::GCContext* gcx) {
MOZ_ASSERT(compartments().empty());
JSRuntime* rt = gcx->runtime(); if (auto callback = rt->destroyZoneCallback) {
callback(gcx, this);
} // Bug 1560019: Malloc memory associated with a zone but not with a specific // GC thing is not currently tracked.
gcx->deleteUntracked(this);
gcx->runtime()->gc.stats().sweptZone();
}
/* * It's simpler if we preserve the invariant that every zone (except atoms * zones) has at least one compartment, and every compartment has at least one * realm. If we know we're deleting the entire zone, then sweepCompartments is * allowed to delete all compartments. In this case, |keepAtleastOne| is false. * If any cells remain alive in the zone, set |keepAtleastOne| true to prohibit * sweepCompartments from deleting every compartment. Instead, it preserves an * arbitrary compartment in the zone.
*/ void Zone::sweepCompartments(JS::GCContext* gcx, bool keepAtleastOne, bool destroyingRuntime) {
MOZ_ASSERT_IF(!isAtomsZone(), !compartments().empty());
MOZ_ASSERT_IF(destroyingRuntime, !keepAtleastOne);
/* * Don't delete the last compartment and realm if keepAtleastOne is * still true, meaning all the other compartments were deleted.
*/ bool keepAtleastOneRealm = read == end && keepAtleastOne;
comp->sweepRealms(gcx, keepAtleastOneRealm, destroyingRuntime);
/* * Don't delete the last realm if keepAtleastOne is still true, meaning * all the other realms were deleted.
*/ bool dontDelete = read == end && keepAtleastOne; if ((realm->marked() || dontDelete) && !destroyingRuntime) {
*write++ = realm;
keepAtleastOne = false;
} else {
realm->destroy(gcx);
}
}
realms().shrinkTo(write - realms().begin());
MOZ_ASSERT_IF(keepAtleastOne, !realms().empty());
MOZ_ASSERT_IF(destroyingRuntime, realms().empty());
}
bool GCRuntime::shouldPreserveJITCode(Realm* realm, const TimeStamp& currentTime,
JS::GCReason reason, bool canAllocateMoreCode, bool isActiveCompartment) { // During shutdown, we must clean everything up, for the sake of leak // detection. if (isShutdownGC()) { returnfalse;
}
// A shrinking GC is trying to clear out as much as it can, and so we should // not preserve JIT code here! if (isShrinkingGC()) { returnfalse;
}
// We are close to our allocatable code limit, so let's try to clean it out. if (!canAllocateMoreCode) { returnfalse;
}
// The topmost frame of JIT code is in this compartment, and so we should // try to preserve this zone's code. if (isActiveCompartment) { returntrue;
}
// The gcPreserveJitCode testing function was used. if (alwaysPreserveCode) { returntrue;
}
// This realm explicitly requested we try to preserve its JIT code. if (realm->preserveJitCode()) { returntrue;
}
// If we're currently animating, and we've already discarded code recently // we can preserve jit code; however we shouldn't hold onto JIT code forever // during animation. if (IsCurrentlyAnimating(realm->lastAnimationTime, currentTime) &&
DiscardedCodeRecently(realm->zone(), currentTime)) { returntrue;
}
// GC Invoked via a testing function. if (reason == JS::GCReason::DEBUG_GC) { returntrue;
}
returnfalse;
}
#ifdef DEBUG class CompartmentCheckTracer final : public JS::CallbackTracer { void onChild(JS::GCCellPtr thing, constchar* name) override; bool edgeIsInCrossCompartmentMap(JS::GCCellPtr dst);
staticbool InCrossCompartmentMap(JSRuntime* rt, JSObject* src,
JS::GCCellPtr dst) { // Cross compartment edges are either in the cross compartment map or in a // debugger weakmap.
Compartment* srccomp = src->compartment();
if (dst.is<JSObject>()) { if (ObjectWrapperMap::Ptr p = srccomp->lookupWrapper(&dst.as<JSObject>())) { if (*p->value().unsafeGet() == src) { returntrue;
}
}
}
if (DebugAPI::edgeIsInDebuggerWeakmap(rt, src, dst)) { returntrue;
}
for (auto i = zone->cellIterUnsafe<JSObject>(kind, empty); !i.done();
i.next()) {
JSFunction* fun = &i->as<JSFunction>(); // When iterating over the GC-heap, we may encounter function objects that // are incomplete (missing a BaseScript when we expect one). We must check // for this case before we can call JSFunction::hasBytecode(). if (fun->isIncomplete()) { continue;
} if (fun->hasBytecode()) {
fun->maybeRelazify(rt);
}
}
}
staticbool ShouldCollectZone(Zone* zone, JS::GCReason reason) { // If we are repeating a GC because we noticed dead compartments haven't // been collected, then only collect zones containing those compartments. if (reason == JS::GCReason::COMPARTMENT_REVIVED) { for (CompartmentsInZoneIter comp(zone); !comp.done(); comp.next()) { if (comp->gcState.scheduledForDestruction) { returntrue;
}
}
returnfalse;
}
// Otherwise we only collect scheduled zones. return zone->isGCScheduled();
}
bool GCRuntime::prepareZonesForCollection(JS::GCReason reason, bool* isFullOut) { #ifdef DEBUG /* Assert that zone state is as we expect */ for (ZonesIter zone(this, WithAtoms); !zone.done(); zone.next()) {
MOZ_ASSERT(!zone->isCollecting());
MOZ_ASSERT_IF(!zone->isAtomsZone(), !zone->compartments().empty()); for (auto i : AllAllocKinds()) {
MOZ_ASSERT(zone->arenas.collectingArenaList(i).isEmpty());
}
} #endif
*isFullOut = true; bool any = false;
for (ZonesIter zone(this, WithAtoms); !zone.done(); zone.next()) { /* Set up which zones will be collected. */ bool shouldCollect = ShouldCollectZone(zone, reason); if (shouldCollect) {
any = true;
zone->changeGCState(Zone::NoGC, Zone::Prepare);
} else {
*isFullOut = false;
}
zone->setWasCollected(shouldCollect);
}
/* Check that at least one zone is scheduled for collection. */ return any;
}
// Update JIT Code state for GC: A few different actions are combined here to // minimize the number of iterations over zones & scripts that required. void GCRuntime::maybeDiscardJitCodeForGC() {
size_t nurserySiteResetCount = 0;
size_t pretenuredSiteResetCount = 0;
// We may need to reset allocation sites and discard JIT code to recover if // we find object lifetimes have changed.
PretenuringZone& pz = zone->pretenuring; bool resetNurserySites = pz.shouldResetNurseryAllocSites(); bool resetPretenuredSites = pz.shouldResetPretenuredAllocSites();
void GCRuntime::purgePropMapTablesForShrinkingGC() {
gcstats::AutoPhase ap(stats(), gcstats::PhaseKind::PURGE_PROP_MAP_TABLES); for (GCZonesIter zone(this); !zone.done(); zone.next()) { if (!canRelocateZone(zone) || zone->keepPropMapTables()) { continue;
}
// Note: CompactPropMaps never have a table. for (auto map = zone->cellIterUnsafe<NormalPropMap>(); !map.done();
map.next()) { if (map->asLinked()->hasTable()) {
map->asLinked()->purgeTable(rt->gcContext());
}
} for (auto map = zone->cellIterUnsafe<DictionaryPropMap>(); !map.done();
map.next()) { if (map->asLinked()->hasTable()) {
map->asLinked()->purgeTable(rt->gcContext());
}
}
}
}
// The debugger keeps track of the URLs for the sources of each realm's scripts. // These URLs are purged on shrinking GCs. void GCRuntime::purgeSourceURLsForShrinkingGC() {
gcstats::AutoPhase ap(stats(), gcstats::PhaseKind::PURGE_SOURCE_URLS); for (GCZonesIter zone(this); !zone.done(); zone.next()) { // URLs are not tracked for realms in the system zone. if (!canRelocateZone(zone) || zone->isSystemZone()) { continue;
} for (CompartmentsInZoneIter comp(zone); !comp.done(); comp.next()) { for (RealmsInCompartmentIter realm(comp); !realm.done(); realm.next()) {
GlobalObject* global = realm.get()->unsafeUnbarrieredMaybeGlobal(); if (global) {
global->clearSourceURLSHolder();
}
}
}
}
}
void GCRuntime::unmarkWeakMaps() { for (GCZonesIter zone(this); !zone.done(); zone.next()) { /* Unmark all weak maps in the zones being collected. */
WeakMapBase::unmarkZone(zone);
}
}
if (!prepareZonesForCollection(reason, &isFull.ref())) { returnfalse;
}
/* * Start a parallel task to clear all mark state for the zones we are * collecting. This is linear in the size of the heap we are collecting and so * can be slow. This usually happens concurrently with the mutator and GC * proper does not start until this is complete.
*/
unmarkTask.initZones(); if (useBackgroundThreads) {
unmarkTask.start();
} else {
unmarkTask.runFromMainThread();
}
/* * Process any queued source compressions during the start of a major * GC. * * Bug 1650075: When we start passing GCOptions::Shutdown for * GCReason::XPCONNECT_SHUTDOWN GCs we can remove the extra check.
*/ if (!isShutdownGC() && reason != JS::GCReason::XPCONNECT_SHUTDOWN) {
StartHandlingCompressionsOnGC(rt);
}
// We can't safely iterate the zones vector from another thread so we copy the // zones to be collected into another vector.
AutoEnterOOMUnsafeRegion oomUnsafe; for (GCZonesIter zone(gc); !zone.done(); zone.next()) { if (!zones.append(zone.get())) {
oomUnsafe.crash("BackgroundUnmarkTask::initZones");
}
for (GCZonesIter zone(this); !zone.done(); zone.next()) {
zone->setPreservingCode(false);
}
// Discard JIT code more aggressively if the process is approaching its // executable code limit. bool canAllocateMoreCode = jit::CanLikelyAllocateMoreExecutableMemory(); auto currentTime = TimeStamp::Now();
/* Clear mark state for WeakMaps in parallel with other work. */
AutoRunParallelTask unmarkWeakMaps(this, &GCRuntime::unmarkWeakMaps,
gcstats::PhaseKind::UNMARK_WEAKMAPS,
GCUse::Unspecified, helperLock);
AutoUnlockHelperThreadState unlock(helperLock);
maybeDiscardJitCodeForGC();
/* * We must purge the runtime at the beginning of an incremental GC. The * danger if we purge later is that the snapshot invariant of * incremental GC will be broken, as follows. If some object is * reachable only through some cache (say the dtoaCache) then it will * not be part of the snapshot. If we purge after root marking, then * the mutator could obtain a pointer to the object and start using * it. This object might never be marked, so a GC hazard would exist.
*/
purgeRuntime();
}
// This will start background free for lifo blocks queued by purgeRuntime, // even if there's nothing in the nursery. Record the number of the minor GC // so we can check whether we need to wait for it to finish or whether a // subsequent minor GC already did this.
collectNurseryFromMajorGC(reason);
initialMinorGCNumber = minorGCNumber;
{
gcstats::AutoPhase ap(stats(), gcstats::PhaseKind::PREPARE); // Relazify functions after discarding JIT code (we can't relazify functions // with JIT code) and before the actual mark phase, so that the current GC // can collect the JSScripts we're unlinking here. We do this only when // we're performing a shrinking GC, as too much relazification can cause // performance issues when we have to reparse the same functions over and // over. if (isShrinkingGC()) {
relazifyFunctionsForShrinkingGC();
purgePropMapTablesForShrinkingGC();
purgeSourceURLsForShrinkingGC();
}
if (isShutdownGC()) { /* Clear any engine roots that may hold external data live. */ for (GCZonesIter zone(this); !zone.done(); zone.next()) {
zone->clearRootsForShutdownGC();
}
AutoUpdateLiveCompartments::~AutoUpdateLiveCompartments() { for (GCCompartmentsIter c(gc->rt); !c.done(); c.next()) { if (c->gcState.hasMarkedCells) {
c->gcState.maybeAlive = true;
}
}
}
Zone::GCState Zone::initialMarkingState() const { if (isAtomsZone()) { // Don't delay gray marking in the atoms zone like we do in other zones. return MarkBlackAndGray;
}
// This is the slice we actually start collecting. The number can be used to // check whether a major GC has started so we must not increment it until we // get here.
incMajorGcNumber();
{
BufferAllocator::MaybeLock lock; for (GCZonesIter zone(this); !zone.done(); zone.next()) { // In an incremental GC, clear the arena free lists to ensure that // subsequent allocations refill them and end up marking new cells black. // See arenaAllocatedDuringGC().
zone->arenas.clearFreeLists();
#ifdef JS_GC_ZEAL if (hasZealMode(ZealMode::YieldBeforeRootMarking)) { for (auto kind : AllAllocKinds()) { for (ArenaIter arena(zone, kind); !arena.done(); arena.next()) {
arena->checkNoMarkedCells();
}
}
} #endif
// Incremental marking barriers are enabled at this point.
zone->changeGCState(Zone::Prepare, zone->initialMarkingState());
// Merge arenas allocated during the prepare phase, then move all arenas // to the collecting arena lists.
zone->arenas.mergeArenasFromCollectingLists();
zone->arenas.moveArenasToCollectingLists();
// Prepare sized allocator for major GC.
zone->bufferAllocator.startMajorCollection(lock);
for (RealmsInZoneIter realm(zone); !realm.done(); realm.next()) {
realm->clearAllocatedDuringGC();
}
}
}
/* * This code ensures that if a compartment is "dead", then it will be * collected in this GC. A compartment is considered dead if its maybeAlive * flag is false. The maybeAlive flag is set if: * * (1) the compartment has been entered (set in beginMarkPhase() above) * (2) the compartment's zone is not being collected (set in * endPreparePhase() above) * (3) an object in the compartment was marked during root marking, either * as a black root or a gray root. This is arranged by * SetCompartmentHasMarkedCells and AutoUpdateLiveCompartments. * (4) the compartment has incoming cross-compartment edges from another * compartment that has maybeAlive set (set by this method). * (5) the compartment has the invisibleToDebugger flag set, as it is * presumed to be a system compartment (set in endPreparePhase() above) * * If the maybeAlive is false, then we set the scheduledForDestruction flag. * At the end of the GC, we look for compartments where * scheduledForDestruction is true. These are compartments that were somehow * "revived" during the incremental GC. If any are found, we do a special, * non-incremental GC of those compartments to try to collect them. * * Compartments can be revived for a variety of reasons, including: * * (1) A dead reflector can be revived by DOM code that still refers to the * underlying DOM node (see bug 811587). * (2) JS_TransplantObject iterates over all compartments, live or dead, and * operates on their objects. This can trigger read barriers and mark * unreachable objects. See bug 803376 for details on this problem. To * avoid the problem, we try to avoid allocation and read barriers * during JS_TransplantObject and the like. * (3) Read barriers. A compartment may only have weak roots and reading one * of these will cause the compartment to stay alive even though the GC * thought it should die. An example of this is Gecko's unprivileged * junk scope, which is handled by ignoring system compartments (see bug * 1868437).
*/
// Propagate the maybeAlive flag via cross-compartment edges.
// Update memory counters for the zones we are collecting. for (GCZonesIter zone(this); !zone.done(); zone.next()) {
zone->updateSchedulingStateOnGCStart();
}
}
#ifdefined(DEBUG) || defined(JS_OOM_BREAKPOINT) // OOM testing limits the engine to using a single helper thread. if (oom::simulator.targetThread() == THREAD_TYPE_GCPARALLEL) { returnfalse;
} #endif
bool GCRuntime::initParallelMarking() { // This is called at the start of collection.
MOZ_ASSERT(canMarkInParallel());
// Reserve/release helper threads for worker runtimes. These are released at // the end of sweeping. If there are not enough helper threads because // other runtimes are marking in parallel then parallel marking will not be // used. if (!rt->isMainRuntime() && !reserveMarkingThreads(markers.length())) { returnfalse;
}
// Allocate stack for parallel markers. The first marker always has stack // allocated. Other markers have their stack freed in // GCRuntime::finishCollection. for (size_t i = 1; i < markers.length(); i++) { if (!markers[i]->initStack()) { returnfalse;
}
}
returntrue;
}
IncrementalProgress GCRuntime::markUntilBudgetExhausted(
SliceBudget& sliceBudget, ParallelMarking allowParallelMarking,
ShouldReportMarkTime reportTime) { // Run a marking slice and return whether the stack is now empty.
AutoMajorGCProfilerEntry s(this);
if (initialState != State::Mark) {
sliceBudget.forceCheck(); if (sliceBudget.isOverBudget()) { return NotFinished;
}
}
// If the queue wants to be gray marking, but we've pushed a black object // since set-color-gray was processed, then we can't switch to gray and must // again wait until gray marking is possible. // // Remove this code if the restriction against marking gray during black is // relaxed. if (queueMarkColor == mozilla::Some(MarkColor::Gray) &&
marker().hasBlackEntries()) { return QueueSuspended;
}
// If the queue wants to be marking a particular color, switch to that color. // In any case, restore the mark color to whatever it was when we entered // this function. bool willRevertToGray = marker().markColor() == MarkColor::Gray;
AutoSetMarkColor autoRevertColor(
marker(), queueMarkColor.valueOr(marker().markColor()));
// Process the mark queue by taking each object in turn, pushing it onto the // mark stack, and processing just the top element with processMarkStackTop // without recursing into reachable objects. while (queuePos < testMarkQueue.length()) {
Value val = testMarkQueue[queuePos++].get(); if (val.isObject()) {
JSObject* obj = &val.toObject();
JS::Zone* zone = obj->zone(); if (!zone->isGCMarking() || obj->isMarkedAtLeast(marker().markColor())) { continue;
}
// If we have started sweeping, obey sweep group ordering. But note that // we will first be called during the initial sweep slice, when the sweep // group indexes have not yet been computed. In that case, we can mark // freely. if (state() == State::Sweep && initialState != State::Sweep) { if (zone->gcSweepGroupIndex < getCurrentSweepGroupIndex()) { // Too late. This must have been added after we started collecting, // and we've already processed its sweep group. Skip it. continue;
} if (zone->gcSweepGroupIndex > getCurrentSweepGroupIndex()) { // Not ready yet. Wait until we reach the object's sweep group.
queuePos--; return QueueSuspended;
}
}
if (marker().markColor() == MarkColor::Gray &&
zone->isGCMarkingBlackOnly()) { // Have not yet reached the point where we can mark this object, so // continue with the GC.
queuePos--; return QueueSuspended;
}
if (marker().markColor() == MarkColor::Black && willRevertToGray) { // If we put any black objects on the stack, we wouldn't be able to // return to gray marking. So delay the marking until we're back to // black marking.
queuePos--; return QueueSuspended;
}
// Mark the object. if (!marker().markOneObjectForTest(obj)) { // If we overflowed the stack here and delayed marking, then we won't be // testing what we think we're testing.
MOZ_ASSERT(obj->asTenured().arena()->onDelayedMarkingList());
printf_stderr( "Hit mark stack limit while marking test queue; test results may " "be invalid");
}
} elseif (val.isString()) {
JSLinearString* str = &val.toString()->asLinear(); if (js::StringEqualsLiteral(str, "yield") && isIncrementalGc()) { return QueueYielded;
}
if (js::StringEqualsLiteral(str, "enter-weak-marking-mode") ||
js::StringEqualsLiteral(str, "abort-weak-marking-mode")) { if (marker().isRegularMarking()) { // We can't enter weak marking mode at just any time, so instead // we'll stop processing the queue and continue on with the GC. Once // we enter weak marking mode, we can continue to the rest of the // queue. Note that we will also suspend for aborting, and then abort // the earliest following weak marking mode.
queuePos--; return QueueSuspended;
} if (js::StringEqualsLiteral(str, "abort-weak-marking-mode")) {
marker().abortLinearWeakMarking();
}
} elseif (js::StringEqualsLiteral(str, "drain")) { auto unlimited = SliceBudget::unlimited();
MOZ_RELEASE_ASSERT(
marker().markUntilBudgetExhausted(unlimited, DontReportMarkTime));
} elseif (js::StringEqualsLiteral(str, "set-color-gray")) {
queueMarkColor = mozilla::Some(MarkColor::Gray); if (state() != State::Sweep || marker().hasBlackEntries()) { // Cannot mark gray yet, so continue with the GC.
queuePos--; return QueueSuspended;
}
marker().setMarkColor(MarkColor::Gray);
} elseif (js::StringEqualsLiteral(str, "set-color-black")) {
queueMarkColor = mozilla::Some(MarkColor::Black);
marker().setMarkColor(MarkColor::Black);
} elseif (js::StringEqualsLiteral(str, "unset-color")) {
queueMarkColor.reset();
}
}
} #endif
for (GCZonesIter zone(this); !zone.done(); zone.next()) { if (zone->nurseryStringsDisabled || zone->nurseryBigIntsDisabled) { // We may need to reset allocation sites and discard JIT code to recover // if we find object lifetimes have changed. if (zone->pretenuring.shouldResetPretenuredAllocSites()) {
zone->unknownAllocSite(JS::TraceKind::String)->maybeResetState();
zone->unknownAllocSite(JS::TraceKind::BigInt)->maybeResetState(); if (zone->nurseryStringsDisabled) {
zone->nurseryStringsDisabled = false;
zonesWhereStringsEnabled++;
} if (zone->nurseryBigIntsDisabled) {
zone->nurseryBigIntsDisabled = false;
zonesWhereBigIntsEnabled++;
}
nursery().updateAllocFlagsForZone(zone);
}
}
}
if (nursery().reportPretenuring()) { if (zonesWhereStringsEnabled) {
fprintf(stderr, "GC re-enabled nursery string allocation in %zu zones\n",
zonesWhereStringsEnabled);
} if (zonesWhereBigIntsEnabled) {
fprintf(stderr, "GC re-enabled nursery big int allocation in %zu zones\n",
zonesWhereBigIntsEnabled);
}
}
}
void GCRuntime::updateAllocationRates() { // Calculate mutator time since the last update. This ignores the fact that // the zone could have been created since the last update.
TimeStamp currentTime = TimeStamp::Now();
TimeDuration totalTime = currentTime - lastAllocRateUpdateTime; if (collectorTimeSinceAllocRateUpdate >= totalTime) { // It shouldn't happen but occasionally we see collector time being larger // than total time. Skip the update in that case. return;
}
// Drop as much work as possible from an ongoing incremental GC so // we can start a new GC after it has finished. if (incrementalState == State::NotActive) { return IncrementalResult::Ok;
}
switch (incrementalState) { case State::NotActive: case State::Finish:
MOZ_CRASH("Unexpected GC state in resetIncrementalGC"); break;
case State::Prepare:
unmarkTask.cancelAndWait();
[[fallthrough]];
case State::MarkRoots: // We haven't done any marking yet at this point. for (GCZonesIter zone(this); !zone.done(); zone.next()) {
zone->changeGCState(zone->gcState(), Zone::NoGC);
zone->clearGCSliceThresholds();
zone->arenas.clearFreeLists();
zone->arenas.mergeArenasFromCollectingLists();
}
case State::Sweep: { // Finish sweeping the current sweep group, then abort. for (CompartmentsIter c(rt); !c.done(); c.next()) {
c->gcState.scheduledForDestruction = false;
}
case State::Finalize: {
isCompacting = false; break;
}
case State::Compact: { // Skip any remaining zones that would have been compacted.
MOZ_ASSERT(isCompacting);
startedCompacting = true;
zonesToMaybeCompact.ref().clear(); break;
}
case State::Decommit: { break;
}
}
stats().reset(reason);
return IncrementalResult::ResetIncremental;
}
AutoDisableBarriers::AutoDisableBarriers(GCRuntime* gc) : gc(gc) { /* * Clear needsIncrementalBarrier early so we don't do any write barriers * during sweeping.
*/ for (GCZonesIter zone(gc); !zone.done(); zone.next()) { if (zone->isGCMarking()) {
MOZ_ASSERT(zone->needsIncrementalBarrier());
zone->setNeedsIncrementalBarrier(false);
}
MOZ_ASSERT(!zone->needsIncrementalBarrier());
}
}
AutoDisableBarriers::~AutoDisableBarriers() { for (GCZonesIter zone(gc); !zone.done(); zone.next()) {
MOZ_ASSERT(!zone->needsIncrementalBarrier()); if (zone->isGCMarking()) {
zone->setNeedsIncrementalBarrier(true);
}
}
}
staticbool ShouldPauseMutatorWhileWaiting(const SliceBudget& budget,
JS::GCReason reason, bool budgetWasIncreased) { // When we're nearing the incremental limit at which we will finish the // collection synchronously, pause the main thread if there is only background // GC work happening. This allows the GC to catch up and avoid hitting the // limit. return budget.isTimeBudget() &&
(reason == JS::GCReason::ALLOC_TRIGGER ||
reason == JS::GCReason::TOO_MUCH_MALLOC) &&
budgetWasIncreased;
}
#ifdef JS_GC_ZEAL // Do the incremental collection type specified by zeal mode if the collection // was triggered by runDebugGC() and incremental GC has not been cancelled by // resetIncrementalGC().
useZeal = isIncremental && reason == JS::GCReason::DEBUG_GC; #endif
if (useZeal && zealModeControlsYieldPoint()) { // Yields between slices occurs at predetermined points in these modes; the // budget is not used. |isIncremental| is still true.
stats().log("Using unlimited budget for two-slice zeal mode");
budget = SliceBudget::unlimited();
}
if (useZeal && hasZealMode(ZealMode::YieldBeforeRootMarking)) { break;
}
[[fallthrough]];
case State::Prepare: if (waitForBackgroundTask(unmarkTask, budget, shouldPauseMutator) ==
NotFinished) { break;
}
incrementalState = State::MarkRoots;
if (isIncremental && initialState == State::Prepare &&
reason == JS::GCReason::BG_TASK_FINISHED) { // The next slice may be long so wait for the embedding to schedule it // rather than doing it as soon as unmarking finishes. This can happen // when the embedding's GC callback sees this slice end with work // available.
MOZ_ASSERT(hasForegroundWork()); break;
}
if (useZeal && hasZealMode(ZealMode::YieldBeforeMarking) &&
isIncremental) { break;
}
[[fallthrough]];
case State::Mark: if (mightSweepInThisSlice(budget.isUnlimited())) {
prepareForSweepSlice(reason);
// Incremental marking validation re-runs all marking non-incrementally, // which requires collecting the nursery. If that might happen in this // slice, do it now while it's safe to do so. if (isIncremental &&
hasZealMode(ZealMode::IncrementalMarkingValidator)) {
collectNurseryFromMajorGC(JS::GCReason::EVICT_NURSERY);
}
}
/* * There are a number of reasons why we break out of collection here, * either ending the slice or to run a new interation of the loop in * GCRuntime::collect()
*/
/* * In incremental GCs where we have already performed more than one * slice we yield after marking with the aim of starting the sweep in * the next slice, since the first slice of sweeping can be expensive. * * This is modified by the various zeal modes. We don't yield in * YieldBeforeMarking mode and we always yield in YieldBeforeSweeping * mode. * * We will need to mark anything new on the stack when we resume, so * we stay in Mark state.
*/ if (isIncremental && !lastMarkSlice) { if ((initialState == State::Mark &&
!(useZeal && hasZealMode(ZealMode::YieldBeforeMarking))) ||
(useZeal && hasZealMode(ZealMode::YieldBeforeSweeping))) {
lastMarkSlice = true;
stats().log("Yielding before starting sweeping"); break;
}
}
// Ensure freeing of nursery owned sized allocations from the initial // minor GC has finished.
MOZ_ASSERT(minorGCNumber >= initialMinorGCNumber); if (minorGCNumber == initialMinorGCNumber) {
MOZ_ASSERT(nursery().sweepTaskIsIdle());
waitBackgroundFreeEnd();
}
{ // Sweep the zones list now that background finalization is finished to // remove and free dead zones, compartments and realms.
gcstats::AutoPhase ap1(stats(), gcstats::PhaseKind::SWEEP);
gcstats::AutoPhase ap2(stats(), gcstats::PhaseKind::DESTROY);
sweepZones(rt->gcContext(), destroyingRuntime);
}
bool GCRuntime::hasForegroundWork() const { switch (incrementalState) { case State::NotActive: // Incremental GC is not running and no work is pending. returnfalse; case State::Prepare: // We yield in the Prepare state after starting unmarking. return !unmarkTask.wasStarted(); case State::Finalize: // We yield in the Finalize state to wait for background sweeping. return !isBackgroundSweeping(); case State::Decommit: // We yield in the Decommit state to wait for background decommit. return !decommitTask.wasStarted(); default: // In all other states there is still work to do. returntrue;
}
}
IncrementalProgress GCRuntime::waitForBackgroundTask(GCParallelTask& task, const SliceBudget& budget, bool shouldPauseMutator) { // Wait here in non-incremental collections, or if we want to pause the // mutator to let the GC catch up. if (budget.isUnlimited() || shouldPauseMutator) {
gcstats::AutoPhase ap(stats(), gcstats::PhaseKind::WAIT_BACKGROUND_THREAD);
Maybe<TimeStamp> deadline; if (budget.isTimeBudget()) {
deadline.emplace(budget.deadline());
}
task.join(deadline);
}
// In incremental collections, yield if the task has not finished and request // a slice to notify us when this happens. if (!budget.isUnlimited()) {
AutoLockHelperThreadState lock; if (task.wasStarted(lock)) {
requestSliceAfterBackgroundTask = true; return NotFinished;
}
// Reset any in progress incremental GC if this was triggered via the // API. This isn't required for correctness, but sometimes during tests // the caller expects this GC to collect certain objects, and we need // to make sure to collect everything possible. if (reason != JS::GCReason::ALLOC_TRIGGER) { return resetIncrementalGC(GCAbortReason::NonIncrementalRequested);
}
// Return true if the budget is actually extended after rounding. staticbool ExtendBudget(SliceBudget& budget, double newDuration) { long millis = lround(newDuration); if (millis <= budget.timeBudget()) { returnfalse;
}
bool GCRuntime::maybeIncreaseSliceBudgetForLongCollections(
SliceBudget& budget, TimeStamp sliceStartTime, TimeStamp gcStartTime) { // For long-running collections, enforce a minimum time budget that increases // linearly with time up to a maximum.
// All times are in milliseconds. struct BudgetAtTime { double time; double budget;
}; const BudgetAtTime MinBudgetStart{1500, 0.0}; const BudgetAtTime MinBudgetEnd{2500, 100.0};
bool GCRuntime::maybeIncreaseSliceBudgetForUrgentCollections(
SliceBudget& budget) { // Enforce a minimum time budget based on how close we are to the incremental // limit.
if (minBytesRemaining < tunables.urgentThresholdBytes() &&
minBytesRemaining != 0) { // Increase budget based on the reciprocal of the fraction remaining. double fractionRemaining = double(minBytesRemaining) / double(tunables.urgentThresholdBytes()); double minBudget = double(defaultSliceBudgetMS()) / fractionRemaining; return ExtendBudget(budget, minBudget);
}
returnfalse;
}
staticvoid ScheduleZones(GCRuntime* gc, JS::GCReason reason) { for (ZonesIter zone(gc, WithAtoms); !zone.done(); zone.next()) { // Re-check heap threshold for alloc-triggered zones that were not // previously collected. Now we have allocation rate data, the heap limit // may have been increased beyond the current size. if (gc->tunables.balancedHeapLimitsEnabled() && zone->isGCScheduled() &&
zone->smoothedCollectionRate.ref().isNothing() &&
reason == JS::GCReason::ALLOC_TRIGGER &&
zone->gcHeapSize.bytes() < zone->gcHeapThreshold.startBytes()) {
zone->unscheduleGC(); // May still be re-scheduled below.
}
if (gc->isShutdownGC()) {
zone->scheduleGC();
}
if (!gc->isPerZoneGCEnabled()) {
zone->scheduleGC();
}
// To avoid resets, continue to collect any zones that were being // collected in a previous slice. if (gc->isIncrementalGCInProgress() && zone->wasGCStarted()) {
zone->scheduleGC();
}
// This is a heuristic to reduce the total number of collections. bool inHighFrequencyMode = gc->schedulingState.inHighFrequencyGCMode(); if (zone->gcHeapSize.bytes() >=
zone->gcHeapThreshold.eagerAllocTrigger(inHighFrequencyMode) ||
zone->mallocHeapSize.bytes() >=
zone->mallocHeapThreshold.eagerAllocTrigger(inHighFrequencyMode) ||
zone->jitHeapSize.bytes() >= zone->jitHeapThreshold.startBytes()) {
zone->scheduleGC();
}
}
}
if (gcCallbackDepth == 0) { // Save scheduled zone information in case the callback clears it. for (ZonesIter zone(this, WithAtoms); !zone.done(); zone.next()) {
zone->gcScheduledSaved_ = zone->gcScheduled_;
}
}
// Save and clear GC options and state in case the callback reenters GC.
JS::GCOptions options = gcOptions();
maybeGcOptions = Nothing(); bool savedFullGCRequested = fullGCRequested;
fullGCRequested = false;
// Restore the original GC options.
maybeGcOptions = Some(options);
// At the end of a GC, clear out the fullGCRequested state. At the start, // restore the previous setting.
fullGCRequested = (status == JSGC_END) ? false : savedFullGCRequested;
if (gcCallbackDepth == 0) { // Ensure any zone that was originally scheduled stays scheduled. for (ZonesIter zone(this, WithAtoms); !zone.done(); zone.next()) {
zone->gcScheduled_ = zone->gcScheduled_ || zone->gcScheduledSaved_;
}
}
}
/* * We disable inlining to ensure that the bottom of the stack with possible GC * roots recorded in MarkRuntime excludes any pointers we use during the marking * implementation.
*/
MOZ_NEVER_INLINE GCRuntime::IncrementalResult GCRuntime::gcCycle( bool nonincrementalByAPI, const SliceBudget& budgetArg,
JS::GCReason reason) { // Assert if this is a GC unsafe region.
rt->mainContextFromOwnThread()->verifyIsSafeToGC();
// It's ok if threads other than the main thread have suppressGC set, as // they are operating on zones which will not be collected from here.
MOZ_ASSERT(!rt->mainContextFromOwnThread()->suppressGC);
// This reason is used internally. See below.
MOZ_ASSERT(reason != JS::GCReason::RESET);
// Background finalization and decommit are finished by definition before we // can start a new major GC. Background allocation may still be running, but // that's OK because chunk pools are protected by the GC lock. bool firstSlice = !isIncrementalGCInProgress(); if (firstSlice) {
assertBackgroundSweepingFinished();
MOZ_ASSERT(decommitTask.isIdle());
}
// Note that GC callbacks are allowed to re-enter GC.
AutoCallGCCallbacks callCallbacks(*this, reason);
// Record GC start time and update global scheduling state.
TimeStamp now = TimeStamp::Now(); if (firstSlice) {
schedulingState.updateHighFrequencyModeOnGCStart(
gcOptions(), lastGCStartTime_, now, tunables);
lastGCStartTime_ = now;
}
schedulingState.updateHighFrequencyModeOnSliceStart(gcOptions(), reason);
// Increase slice budget for long running collections before it is recorded by // AutoGCSlice.
SliceBudget budget(budgetArg); bool budgetWasIncreased =
maybeIncreaseSliceBudget(budget, now, lastGCStartTime_);
ScheduleZones(this, reason);
auto updateCollectorTime = MakeScopeExit([&] { if (const gcstats::Statistics::SliceData* slice = stats().lastSlice()) {
collectorTimeSinceAllocRateUpdate += slice->duration();
}
});
IncrementalResult result =
budgetIncrementalGC(nonincrementalByAPI, reason, budget); if (result == IncrementalResult::ResetIncremental) { if (incrementalState == State::NotActive) { // The collection was reset and has finished. return result;
}
// The collection was reset but we must finish up some remaining work.
reason = JS::GCReason::RESET;
}
#ifdef JS_GC_ZEAL staticbool IsDeterministicGCReason(JS::GCReason reason) { switch (reason) { case JS::GCReason::API: case JS::GCReason::DESTROY_RUNTIME: case JS::GCReason::LAST_DITCH: case JS::GCReason::TOO_MUCH_MALLOC: case JS::GCReason::TOO_MUCH_WASM_MEMORY: case JS::GCReason::TOO_MUCH_JIT_CODE: case JS::GCReason::ALLOC_TRIGGER: case JS::GCReason::DEBUG_GC: case JS::GCReason::CC_FORCED: case JS::GCReason::SHUTDOWN_CC: case JS::GCReason::ABORT_GC: case JS::GCReason::DISABLE_GENERATIONAL_GC: case JS::GCReason::FINISH_GC: case JS::GCReason::PREPARE_FOR_TRACING: returntrue;
// The GC can only clean up scheduledForDestruction realms that were marked live // by a barrier (e.g. by RemapWrappers from a navigation event). It is also // common to have realms held live because they are part of a cycle in gecko, // e.g. involving the HTMLDocument wrapper. In this case, we need to run the // CycleCollector in order to remove these edges before the realm can be freed. void GCRuntime::maybeDoCycleCollection() { conststaticfloat ExcessiveGrayRealms = 0.8f; conststatic size_t LimitGrayRealms = 200;
/* If we attempt to invoke the GC while we are running in the GC, assert. */
MOZ_RELEASE_ASSERT(!JS::RuntimeHeapIsBusy());
}
bool GCRuntime::checkIfGCAllowedInCurrentState(JS::GCReason reason) { if (rt->mainContextFromOwnThread()->suppressGC) { returnfalse;
}
// Only allow shutdown GCs when we're destroying the runtime. This keeps // the GC callback from triggering a nested GC and resetting global state. if (rt->isBeingDestroyed() && !isShutdownGC()) { returnfalse;
}
#ifdef JS_GC_ZEAL if (deterministicOnly && !IsDeterministicGCReason(reason)) { returnfalse;
} #endif
for (CompartmentsIter c(rt); !c.done(); c.next()) { if (c->gcState.scheduledForDestruction) { returntrue;
}
}
returnfalse;
}
struct MOZ_RAII AutoSetZoneSliceThresholds { explicit AutoSetZoneSliceThresholds(GCRuntime* gc) : gc(gc) { // On entry, zones that are already collecting should have a slice threshold // set. for (ZonesIter zone(gc, WithAtoms); !zone.done(); zone.next()) {
MOZ_ASSERT(zone->wasGCStarted() ==
zone->gcHeapThreshold.hasSliceThreshold());
MOZ_ASSERT(zone->wasGCStarted() ==
zone->mallocHeapThreshold.hasSliceThreshold());
}
}
~AutoSetZoneSliceThresholds() { // On exit, update the thresholds for all collecting zones. bool waitingOnBGTask = gc->isWaitingOnBackgroundTask(); for (ZonesIter zone(gc, WithAtoms); !zone.done(); zone.next()) { if (zone->wasGCStarted()) {
zone->setGCSliceThresholds(*gc, waitingOnBGTask);
} else {
MOZ_ASSERT(!zone->gcHeapThreshold.hasSliceThreshold());
MOZ_ASSERT(!zone->mallocHeapThreshold.hasSliceThreshold());
}
}
}
if (!isIncrementalGCInProgress() && tunables.balancedHeapLimitsEnabled()) {
updateAllocationRates();
}
bool repeat; do {
IncrementalResult cycleResult =
gcCycle(nonincrementalByAPI, budget, reason);
if (reason == JS::GCReason::ABORT_GC) {
MOZ_ASSERT(!isIncrementalGCInProgress());
stats().log("GC aborted by request"); break;
}
/* * Sometimes when we finish a GC we need to immediately start a new one. * This happens in the following cases: * - when we reset the current GC * - when finalizers drop roots during shutdown * - when zones that we thought were dead at the start of GC are * not collected (see the large comment in beginMarkPhase)
*/
repeat = false; if (!isIncrementalGCInProgress()) { if (cycleResult == ResetIncremental) {
repeat = true;
} elseif (rootsRemoved && isShutdownGC()) { /* Need to re-schedule all zones for GC. */
JS::PrepareForFullGC(rt->mainContextFromOwnThread());
repeat = true;
reason = JS::GCReason::ROOTS_REMOVED;
} elseif (shouldRepeatForDeadZone(reason)) {
repeat = true;
reason = JS::GCReason::COMPARTMENT_REVIVED;
}
}
} while (repeat);
if (reason == JS::GCReason::COMPARTMENT_REVIVED) {
maybeDoCycleCollection();
}
#ifdef JS_GC_ZEAL if (!isIncrementalGCInProgress()) { if (hasZealMode(ZealMode::CheckHeapAfterGC)) {
gcstats::AutoPhase ap(stats(), gcstats::PhaseKind::TRACE_HEAP);
CheckHeapAfterGC(rt);
} if (hasZealMode(ZealMode::CheckGrayMarking)) {
MOZ_RELEASE_ASSERT(CheckGrayMarkingState(rt));
}
} #endif
stats().log("GC slice ending in state %s", StateName(incrementalState));
UnscheduleZones(this);
}
SliceBudget GCRuntime::defaultBudget(JS::GCReason reason, int64_t millis) { // millis == 0 means use internal GC scheduling logic to come up with // a duration for the slice budget. This may end up still being zero // based on preferences. if (millis == 0) {
millis = defaultSliceBudgetMS();
}
// If the embedding has registered a callback for creating SliceBudgets, // then use it. if (createBudgetCallback) { return createBudgetCallback(reason, millis);
}
// Otherwise, the preference can request an unlimited duration slice. if (millis == 0) { return SliceBudget::unlimited();
}
// If we're not collecting because we're out of memory then skip the // compacting phase if we need to finish an ongoing incremental GC // non-incrementally to avoid janking the browser. if (!IsOOMReason(initialReason)) { if (incrementalState == State::Compact) {
abortGC(); return;
}
if (!ZonesSelected(this)) {
JS::PrepareForIncrementalGC(rt->mainContextFromOwnThread());
}
collect(false, budget, JS::GCReason::DEBUG_GC);
}
void js::PrepareForDebugGC(JSRuntime* rt) { // If zones have already been scheduled then use them. if (ZonesSelected(&rt->gc)) { return;
}
// If we already started a GC then continue with the same set of zones. This // prevents resetting an ongoing GC when new zones are added.
JSContext* cx = rt->mainContextFromOwnThread(); if (JS::IsIncrementalGCInProgress(cx)) {
JS::PrepareForIncrementalGC(cx); return;
}
// Otherwise schedule all zones.
JS::PrepareForFullGC(rt->mainContextFromOwnThread());
}
void GCRuntime::onOutOfMallocMemory() { // Stop allocating new chunks.
allocTask.cancelAndWait();
// Make sure we release anything queued for release.
decommitTask.join();
nursery().joinDecommitTask();
// Wait for background free of nursery huge slots to finish.
sweepTask.join();
void GCRuntime::onOutOfMallocMemory(const AutoLockGC& lock) { #ifdef DEBUG // Release any relocated arenas we may be holding on to, without releasing // the GC lock.
releaseHeldRelocatedArenasWithoutUnlocking(lock); #endif
// Throw away any excess chunks we have lying around.
freeEmptyChunks(lock);
// Immediately decommit as many arenas as possible in the hopes that this // might let the OS scrape together enough pages to satisfy the failing // malloc request. if (DecommitEnabled()) {
decommitFreeArenasWithoutUnlocking(lock);
}
}
// We ignore gcMaxBytes when allocating for minor collection. However, if we // overflowed, we disable the nursery. The next time we allocate, we'll fail // because bytes >= gcMaxBytes. if (heapSize.bytes() >= tunables.gcMaxBytes()) { if (!nursery().isEmpty()) {
nursery().collect(options, JS::GCReason::DISABLE_GENERATIONAL_GC);
MOZ_ASSERT(nursery().isEmpty());
startBackgroundFreeAfterMinorGC();
}
// Disabling the nursery triggers pre-barriers when we discard JIT // code. Normally we don't allow any barriers in a major GC and there is an // assertion to check this in PreWriteBarrier. For the case where we disable // the nursery during a major GC, set up a minor GC session to silence the // assertion.
AutoGCSession session(this, JS::HeapState::MinorCollecting);
nursery().disable();
}
}
void GCRuntime::startBackgroundFreeAfterMinorGC() { // Called after nursery collection. Free whatever blocks are safe to free now.
void js::gc::FinishGC(JSContext* cx, JS::GCReason reason) { // Calling this when GC is suppressed won't have any effect.
MOZ_ASSERT(!cx->suppressGC);
// GC callbacks may run arbitrary code, including JS. Check this regardless of // whether we GC for this invocation.
MOZ_ASSERT(cx->isNurseryAllocAllowed());
if (JS::IsIncrementalGCInProgress(cx)) {
JS::PrepareForIncrementalGC(cx);
JS::FinishIncrementalGC(cx, reason);
}
}
Compartment* comp = nullptr;
Zone* zone = nullptr;
JS::CompartmentSpecifier compSpec =
options.creationOptions().compartmentSpecifier(); switch (compSpec) { case JS::CompartmentSpecifier::NewCompartmentInSystemZone: // systemZone might be null here, in which case we'll make a zone and // set this field below.
zone = rt->gc.systemZone; break; case JS::CompartmentSpecifier::NewCompartmentInExistingZone:
zone = options.creationOptions().zone();
MOZ_ASSERT(zone); break; case JS::CompartmentSpecifier::ExistingCompartment:
comp = options.creationOptions().compartment();
zone = comp->zone(); break; case JS::CompartmentSpecifier::NewCompartmentAndZone: break;
}
bool invisibleToDebugger = options.creationOptions().invisibleToDebugger(); if (comp) { // Debugger visibility is per-compartment, not per-realm, so make sure the // new realm's visibility matches its compartment's.
MOZ_ASSERT(comp->invisibleToDebugger() == invisibleToDebugger);
} else {
compHolder = cx->make_unique<JS::Compartment>(zone, invisibleToDebugger); if (!compHolder) { return nullptr;
}
// Make sure we don't put system and non-system realms in the same // compartment. if (!compHolder) {
MOZ_RELEASE_ASSERT(realm->isSystem() == IsSystemCompartment(comp));
}
AutoLockGC lock(rt);
// Reserve space in the Vectors before we start mutating them. if (!comp->realms().reserve(comp->realms().length() + 1) ||
(compHolder &&
!zone->compartments().reserve(zone->compartments().length() + 1)) ||
(zoneHolder && !rt->gc.zones().reserve(rt->gc.zones().length() + 1))) {
ReportOutOfMemory(cx); return nullptr;
}
// After this everything must be infallible.
comp->realms().infallibleAppend(realm.get());
if (compHolder) {
zone->compartments().infallibleAppend(compHolder.release());
}
if (zoneHolder) {
rt->gc.zones().infallibleAppend(zoneHolder.release());
// Lazily set the runtime's system zone. if (compSpec == JS::CompartmentSpecifier::NewCompartmentInSystemZone) {
MOZ_RELEASE_ASSERT(!rt->gc.systemZone);
MOZ_ASSERT(zone->isSystemZone());
rt->gc.systemZone = zone;
}
}
return realm.release();
}
void GCRuntime::runDebugGC() { #ifdef JS_GC_ZEAL if (rt->mainContextFromOwnThread()->suppressGC) { return;
}
if (hasZealMode(ZealMode::GenerationalGC)) { return minorGC(JS::GCReason::DEBUG_GC);
}
PrepareForDebugGC(rt);
auto budget = SliceBudget::unlimited(); if (hasZealMode(ZealMode::IncrementalMultipleSlices)) { /* * Start with a small slice limit and double it every slice. This * ensure that we get multiple slices, and collection runs to * completion.
*/ if (!isIncrementalGCInProgress()) {
zealSliceBudget = zealFrequency / 2;
} else {
zealSliceBudget *= 2;
}
budget = SliceBudget(WorkBudget(zealSliceBudget));
/* Reset the slice size when we get to the sweep or compact phases. */ if ((initialState == State::Mark && incrementalState == State::Sweep) ||
(initialState == State::Sweep && incrementalState == State::Compact)) {
zealSliceBudget = zealFrequency / 2;
}
} elseif (zealModeControlsYieldPoint()) { // These modes trigger incremental GC that happens in two slices and the // supplied budget is ignored by incrementalSlice.
budget = SliceBudget(WorkBudget(1));
/* * Check that internal hash tables no longer have any pointers to things * that have been moved.
*/
rt->geckoProfiler().checkStringsMapAfterMovingGC(); if (rt->hasJitRuntime() && rt->jitRuntime()->hasInterpreterEntryMap()) {
rt->jitRuntime()->getInterpreterEntryMap()->checkScriptsAfterMovingGC();
} for (ZonesIter zone(this, SkipAtoms); !zone.done(); zone.next()) {
zone->checkUniqueIdTableAfterMovingGC();
zone->shapeZone().checkTablesAfterMovingGC(zone);
zone->checkAllCrossCompartmentWrappersAfterMovingGC();
zone->checkScriptMapsAfterMovingGC();
// Note: CompactPropMaps never have a table.
JS::AutoCheckCannotGC nogc; for (auto map = zone->cellIterUnsafe<NormalPropMap>(); !map.done();
map.next()) { if (PropMapTable* table = map->asLinked()->maybeTable(nogc)) {
table->checkAfterMovingGC(zone);
}
} for (auto map = zone->cellIterUnsafe<DictionaryPropMap>(); !map.done();
map.next()) { if (PropMapTable* table = map->asLinked()->maybeTable(nogc)) {
table->checkAfterMovingGC(zone);
}
}
WeakMapBase::checkWeakMapsAfterMovingGC(zone);
}
for (CompartmentsIter c(this); !c.done(); c.next()) { for (RealmsInCompartmentIter r(c); !r.done(); r.next()) {
r->dtoaCache.checkCacheAfterMovingGC(); if (r->debugEnvs()) {
r->debugEnvs()->checkHashTablesAfterMovingGC();
}
}
}
} #endif
// For use in a debugger. void js::gc::Cell::dump() const {
js::Fprinter out(stderr);
dump(out);
} #endif
JS_PUBLIC_API bool js::gc::detail::CanCheckGrayBits(const TenuredCell* cell) { // We do not check the gray marking state of cells in the following cases: // // 1) When OOM has caused us to clear the gcGrayBitsValid_ flag. // // 2) When we are in an incremental GC and examine a cell that is in a zone // that is not being collected. Gray targets of CCWs that are marked black // by a barrier will eventually be marked black in a later GC slice. // // 3) When mark bits are being cleared concurrently by a helper thread.
// Check that a cell is not marked gray. // // Since this is a debug-only check, take account of the eventual mark state // of cells that will be marked black by the next GC slice in an incremental // GC. For performance reasons we don't do this in CellIsMarkedGrayIfKnown.
// TODO: I'd like to AssertHeapIsIdle() here, but this ends up getting // called during GC and while iterating the heap for memory reporting.
MOZ_ASSERT(!JS::RuntimeHeapIsCycleCollecting());
if (tc->zone()->isGCMarkingBlackAndGray()) { // We are doing gray marking in the cell's zone. Even if the cell is // currently marked gray it may eventually be marked black. Delay checking // non-black cells until we finish gray marking.
if (!tc->isMarkedBlack()) {
JSRuntime* rt = tc->zone()->runtimeFromMainThread();
AutoEnterOOMUnsafeRegion oomUnsafe; if (!rt->gc.cellsToAssertNotGray.ref().append(cell)) {
oomUnsafe.crash("Can't append to delayed gray checks list");
}
} return;
}
template <typename T> void js::gc::ClearEdgesTracer::onEdge(T** thingp, constchar* name) { // We don't handle removing pointers to nursery edges from the store buffer // with this tracer. Check that this doesn't happen.
T* thing = *thingp;
MOZ_ASSERT(!IsInsideNursery(thing));
// Fire the pre-barrier since we're removing an edge from the graph.
InternalBarrierMethods<T*>::preBarrier(thing);
¤ Diese beiden folgenden Angebotsgruppen bietet das Unternehmen0.151Angebot
(Wie Sie bei der Firma Beratungs- und Dienstleistungen beauftragen können 2026-04-26)
¤
Die Informationen auf dieser Webseite wurden
nach bestem Wissen sorgfältig zusammengestellt. Es wird jedoch weder Vollständigkeit, noch Richtigkeit,
noch Qualität der bereit gestellten Informationen zugesichert.
Bemerkung:
Die farbliche Syntaxdarstellung und die Messung sind noch experimentell.
