Export domain drop list for open TLDs to Drive - #3223
Conversation
b41b53d to
9b80bd8
Compare
| static final String CSV_HEADER = "domain_name,tld,deletion_time"; | ||
|
|
||
| private static final String SELECT_UPCOMING_DELETIONS_STATEMENT = | ||
| """ |
There was a problem hiding this comment.
Actually, a better way to do this is to load all Tlds first (in a separate query) and check which ones have XAP enabled as of the current time, and then just filter explicitly to that list of TLDs in the query. Not all open TLDs will necessarily have XAP enabled so that is a better way to do it, and we don't want to facilitate drop-catching on non-XAP-enabled TLDs.
There was a problem hiding this comment.
Done. Evaluated open TLDs that have XAP enabled at clock.now(), and passed them explicitly to the upcoming deletions query as xapTlds.
| StringBuilder csvBuilder = new StringBuilder(); | ||
| csvBuilder.append(CSV_HEADER).append('\n'); | ||
| for (Object[] row : queryResults) { | ||
| csvBuilder.append(row[0]).append(',').append(row[1]).append(',').append(row[2]).append('\n'); |
There was a problem hiding this comment.
Is this really the best way to construct a CSV file? How do we do it elsewhere in the Nomulus codebase?
There was a problem hiding this comment.
Done. Replaced manual string building with org.apache.commons.csv.CSVPrinter (configured with Linux newline record separator).
b5efc1a to
2f22abf
Compare
| } | ||
|
|
||
| Instant now = clock.now(); | ||
| ImmutableSet<String> openTlds = |
There was a problem hiding this comment.
Rename this to xapTlds, since it's not just openTlds.
There was a problem hiding this comment.
Done. Renamed to xapTlds in SELECT_UPCOMING_DELETIONS_STATEMENT and throughout run().
| alpha.open2,open2,2020-02-04T02:02:02Z | ||
| zebra.open1,open1,2020-02-07T02:02:02Z | ||
| """ | ||
| .replace("\n", "\r\n")); |
There was a problem hiding this comment.
I don't like these Windows-style line-endings at all. Configure the Apache CSV writer so that it correctly uses Linux-style newlines consisting of just \n
There was a problem hiding this comment.
Done. Configured CSVFormat.DEFAULT.builder().setRecordSeparator('\n').build() so output strictly uses Linux newlines (\n), and reverted tests to assert against \n.
| void test_emptyDropList_outputsHeaderOnly() throws Exception { | ||
| persistActiveDomain("active.open1"); | ||
| action.run(); | ||
| verifyExportedToDrive("domain_name,tld,deletion_time\r\n"); |
There was a problem hiding this comment.
Yeah there should be no \r in here or anywhere else.
There was a problem hiding this comment.
Done. Removed all \r carriage returns across the codebase and test assertions.
Redo the exported drop list mechanism by replacing the legacy per-TLD mode in ExportDomainListsAction with a dedicated, once-daily ExportDropListAction. Key changes: 1. Reverted ExportDomainListsAction to unconditionally export single-column active registered domains and deprecated the INCLUDE_PENDING_DELETE_DATE_FOR_DOMAINS feature flag. 2. Implemented ExportDropListAction at /_dr/task/exportDropList to query the read replica for upcoming deletions on all open TLDs (where invoicing is enabled) and output an alphabetically sorted CSV file (domain_name,tld,deletion_time) to a designated Google Drive folder. 3. Added domainDropListDriveFolderId configuration setting and provider. 4. Registered the action in RequestComponent, routing.txt, and Cloud Scheduler tasks for production and sandbox. 5. Added comprehensive test coverage in ExportDropListActionTest and cleaned up legacy test cases in ExportDomainListsActionTest. BUG=b/553658111
2f22abf to
a773150
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| static final ImmutableList<String> CSV_HEADER = | ||
| ImmutableList.of("domain_name", "tld", "deletion_time"); | ||
| private static final CSVFormat CSV_FORMAT = | ||
| CSVFormat.DEFAULT.builder().setRecordSeparator('\n').build(); |
2 participants
Redo the exported drop list mechanism by replacing the legacy per-TLD mode in ExportDomainListsAction with a dedicated, once-daily ExportDropListAction.
Key changes:
BUG=b/553658111
This change is