Jump to content

Search Platform/Weekly Updates/2023-04-28

From Wikitech
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

Summary

Annual planning took a reasonable amount of mind share this week. Search Update Pipeline is moving forward. Some unexpected work around operations and development support.

What we've accomplished

Search Analysis

Search Update Pipeline

  • Write a small job that copies the cirrus index data in avro to a smaller dataset written in parquet, will be used to identify inconsistencies between the cirrus index the mysql page table.
  • Upgrade WDQS Streaming Updater to Flink 1.16, testing is done, deployment to production is still pending - https://phabricator.wikimedia.org/T289836

Search SLO

Operations / SRE

  • Hardware requests for next fiscal year

Misc

Wrote a small patch to cleanup error handling of CirrusSearch writes (work initially started by Emmanuel).

  • Tried to figure out what is going on with WCQS/commons next year, if anything.
  • Met with Growth team to talk about product collaboration between reader/editor needs and search, disc, brows. There is a lot of shared thinking and overlap, and hopefully there's some room to collaborate with the team on reader discovery stuff.