Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Export accounts from workspace to csv and add items from drag-n-drop csv #184

Merged
merged 7 commits into from
Dec 9, 2024

Conversation

foysalit
Copy link
Contributor

@foysalit foysalit commented Sep 11, 2024

For record keeping, sharing data for further research and fill legal requests exporting data of a group of users is often done manually by mods. This is aimed to make that process much easier.

Also, allow drag-n-dropping a csv file and filling in workspace items from the csv content.

Screenshot 2024-12-06 at 23 33 32

Export configuration

Screenshot 2024-12-07 at 01 02 00

Workspace import file button small mode

Screenshot 2024-12-07 at 01 02 12

Workspace import file button full mode

Screenshot 2024-12-07 at 01 02 27

Workspace file import error

Comment on lines 156 to 159
repo.email || 'Unknown',
`${repo.ip || 'Unknown'}`,
`${profile?.displayName || 'Unknown'}`,
repo.labels?.map(({ val }) => val).join(', ') || 'None',
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I almost wonder if an empty value could be more useful than a default value here. I suppose it depends how the CSV will be used.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah I think our current use case is for mod folks to get these exports and review manually in which case, I feel like the explicit default values are more helpful.

repo.labels?.map(({ val }) => val).join(', ') || 'None',
buildBlueSkyAppUrl({ did: repo.did }),
]
return line.join(',')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Values containing commas or newlines will end up breaking the CSV format. If we add some escaping we should be able to address it, though:

const escapeCSVValue = (value: string) => {
  if (value.contains(',') || value.contains('"') || value.contains('\r') || value.contains('\n')) {
    return `"${value.replaceAll('"', '""')}"`
  }
  return value
}

// ...

  return line.map(escapeCSVValue).join(',')

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yh except for labels/tags list, we don't have anything that contain those characters though but will add to make it more future proof.

@foysalit foysalit changed the title ✨ Export accounts from workspace to csv ✨ Export accounts from workspace to csv and add items from drag-n-drop csv Dec 6, 2024
Comment on lines +314 to +318
<WorkspaceItemCreator
onFileUploadClick={() => {
dropzoneRef.current?.open()
}}
/>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This uses two different patterns:

  • Data binding through props
  • Data binding through (some sort) of context
    Sadly I don't see an easy way to do "clean" this. If you do, I'd change this, otherwise let's leave it.

lib/csv.ts Outdated
Comment on lines 17 to 33
export function createCSV({
headers,
lines,
filename,
lineDelimiter = '\n',
}: {
lines: string[]
headers: string[]
filename?: string
lineDelimiter?: string
}) {
return {
filename: (filename || Date.now().toString()) + '.csv',
headerRow: headers.join(',') + lineDelimiter, // make your own csv head
body: lines.join(lineDelimiter),
}
}
Copy link
Contributor

@matthieusieben matthieusieben Dec 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The return value does not match the returned type. Is there a particular reason for that ?

The data structure to represent the CSV here is a mix between encoded data (a multi line string of comma separated values, with the firt line being a header) and "parsed" data (internal data structure), Looks like we could benefit from defining and using a type interface (or even a class ?) for internal representation of CSVs:

type CsvContent = {
  headers: string[]
  rows: [][]
  lineDelimiter: "," | ":" | "\t" 
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The return value does not match the returned type

not sure if I follow? the return type is expected to be CsvContent which i left implicit I guess. agreed we can tidy this up a bit more to make a more strict definition but for now the use-case is super low-stake so wanted to keep this flexible/open.

lib/csv.ts Outdated Show resolved Hide resolved
lib/csv.ts Outdated Show resolved Hide resolved
lib/csv.ts Outdated Show resolved Hide resolved
Copy link
Contributor

@matthieusieben matthieusieben left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Some room for refactor, but nothing that needs to be done now.

@arcalinea arcalinea temporarily deployed to account-export - ozone-staging PR #184 December 9, 2024 20:21 — with Render Destroyed
@foysalit
Copy link
Contributor Author

foysalit commented Dec 9, 2024

Jammed in one more feature here to allow profile bio/record content search in workspace filter panel.

@foysalit foysalit merged commit 325581a into main Dec 9, 2024
3 checks passed
@matthieusieben matthieusieben deleted the account-export branch December 10, 2024 08:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants