Swift/Other Processes

This page may be outdated or contain incorrect details. Please update it if you can.

Current

Ganglia Logtailer

See swift::base in puppet for this.

• ganglia-logtailer process running on all swift nodes via cron (front and back ends)
• parses logs to gather performance and throughput information, feeds that into ganglia

swift-ganglia-report-global-stats

See role::swift::pmtpa-prod::ganglia_reporter in puppet for this.

• runs on one server per cluster, currently ms-fe1
• reports cluster-wide statistics to ganglia
  • object count, bytes used, change in # objects
  • these aren't related to a specific server but instead reflect cluster-wide stats

Old

purgethumbs.py

Unused; can't find source to this anywhere; does it still exist?

• purges thumbnails from all media stores
• takes in a list of images to purge and attempts to purge all thumbnails for that list of images from all three locations: ms5, swift, and squid
• used to clean out unreferenced thumbnails generated by ariel's script

delete-containerless-objects

Can't find source to this anywhere; does it still exist?

• script to crawl the object contents in a swift backend, compare this to a list of what's in the container, and delete objects that exist but aren't in the container list.
  • note that this shouldn't happen in the first place

Swift Cleaner

No longer running.

• compared thumbnails on swift to those on ms5. deleted things on swift that didn't match what's on ms5.
• two instances used to run on iron - full scan and incremental scan.
  • incremental took about 2 hours
  • full took about 2 weeks
• was installed in /opt/swiftcleaner. canonical source in git:operations/software/swiftcleaner