extract_archive ignores overwrite parameter, causing stale measure.xml files to persist #858

Summary

extract_archive in dj_jobs/run_simulate_data_point.rb declares overwrite = true as a default parameter but never checks it — the method body always skips existing files, regardless of the argument passed.

Root Cause

def extract_archive(archive_filename, destination, overwrite = true)
  Zip::File.open(archive_filename) do |zf|
    zf.each do |f|
      f_path = File.join(destination, f.name)
      FileUtils.mkdir_p(File.dirname(f_path))
      if File.exist?(f_path)
        @sim_logger.warn "SKIPPED: #{f.name}, already existed."  # always skips — overwrite is never checked
      else
        zf.extract(f, f_path)
      end
    end
  end
end

The overwrite parameter is declared but has no effect.

Impact

If a previous worker died mid-extraction (e.g., killed by a rollout restart), or if an analysis directory was left behind from a prior run, subsequent workers will skip re-extracting files that are already present. This causes measure.xml files from the old run to persist on NFS.

OpenStudio's BCLMeasure loader then rejects the stale/partial measure.xml with errors such as:

• could not be read as XML data
• is not a measure directory
• Workflow is invalid

This resulted in 15 out of 50 datapoints in an analysis failing with datapoint failure status.

Reproduction

1. Start a large analysis on a multi-node cluster
2. Trigger a worker rollout restart while workers are actively downloading/extracting the analysis zip
3. On the next run, workers will find partial analysis directories on NFS and skip extraction of all already-present files
4. Datapoints fail with BCLMeasure XML errors

Fix

Honor the overwrite parameter and pass it through to rubyzip's extract block:

def extract_archive(archive_filename, destination, overwrite = true)
  ::Zip.sort_entries = true
  Zip::File.open(archive_filename) do |zf|
    zf.each do |f|
      @sim_logger.info "Zip: Extracting #{f.name}"
      f_path = File.join(destination, f.name)
      FileUtils.mkdir_p(File.dirname(f_path))
      if File.exist?(f_path) && !overwrite
        @sim_logger.warn "SKIPPED: #{f.name}, already existed."
      else
        zf.extract(f, f_path) { overwrite }  # rubyzip block return value controls overwrite
      end
    end
  end
end

With overwrite = true (the default), all files are always extracted fresh from the zip, replacing any stale content. This is safe because the Redis distributed lock fix (#857) ensures only one worker at a time runs extract_archive for a given analysis.

Notes

• Only dj_jobs/run_simulate_data_point.rb needs to change; the resque job delegates to DjJobs::RunSimulateDataPoint directly and has no independent copy of this method.
• This fix is a prerequisite companion to initialize_worker zip lock uses flock which is not cross-host safe on NFS, causing zlib corruption race on Kubernetes #857 — without it, the Redis lock prevents the race but stale files from pre-reset runs can still survive and cause failures.