Skip to content

feat(console): add JSON export option to database documents #2897

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Divyansh2992 wants to merge 17 commits into
base: main
Choose a base branch
from
Open

feat(console): add JSON export option to database documents #2897

Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
05cbf96
feat(console): add JSON export option to database documents
Divyansh2992 Feb 28, 2026
3a392e9
cleanup: remove redundant export-json wizard
Divyansh2992 Feb 28, 2026
f7a82df
fix: update minimatch override to 10.2.3 to resolve ReDoS vulnerabili…
Divyansh2992 Mar 1, 2026
6474248
style: run prettier to fix formatting and resolve lint errors
Divyansh2992 Mar 1, 2026
848930c
fixed the minor bug
Divyansh2992 Mar 1, 2026
443d244
modified page.svelte
Divyansh2992 Mar 1, 2026
23ce56d
fi the adjustment
Divyansh2992 Mar 1, 2026
0bb84d2
fixed progress bar
Divyansh2992 Mar 1, 2026
eaf458b
fix changes
Divyansh2992 Mar 1, 2026
675cdbd
updated page.svelte
Divyansh2992 Mar 1, 2026
c9ebc6c
fix the error
Divyansh2992 Mar 1, 2026
485804a
Document count in notification
Divyansh2992 Mar 1, 2026
0791978
Merge remote-tracking branch 'upstream/main' into feature/database-js…
Divyansh2992 Mar 5, 2026
0c658f8
fixed the issue
Divyansh2992 Mar 5, 2026
a80dfd2
fixing Hardcoded progress bar colors bypass design system and break d…
Divyansh2992 Mar 5, 2026
f32a9ea
chore: update sass to fix immutable prototype pollution vulnerability…
Divyansh2992 Mar 6, 2026
d0f256c
Resolved the issue
Divyansh2992 Mar 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/lib/actions/analytics.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,7 @@ export enum Click {
DatabaseDatabaseDelete = 'click_database_delete',
DatabaseImportCsv = 'click_database_import_csv',
DatabaseExportCsv = 'click_database_export_csv',
DatabaseExportJson = 'click_database_export_json',
Comment thread
Divyansh2992 marked this conversation as resolved.
Comment thread
Divyansh2992 marked this conversation as resolved.
DomainCreateClick = 'click_domain_create',
DomainDeleteClick = 'click_domain_delete',
DomainRetryDomainVerificationClick = 'click_domain_retry_domain_verification',
Expand Down Expand Up @@ -283,6 +284,7 @@ export enum Submit {
DatabaseUpdateName = 'submit_database_update_name',
DatabaseImportCsv = 'submit_database_import_csv',
DatabaseExportCsv = 'submit_database_export_csv',
DatabaseExportJson = 'submit_database_export_json',
DatabaseBackupDelete = 'submit_database_backup_delete',
DatabaseBackupPolicyCreate = 'submit_database_backup_policy_create',

Expand Down
2 changes: 1 addition & 1 deletion ...sole)/project-[region]-[project]/databases/database-[database]/table-[table]/+page.svelte
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -239,7 +239,7 @@
<Icon icon={IconDownload} size="s" />
</Button>

<svelte:fragment slot="tooltip">Export CSV</svelte:fragment>
<svelte:fragment slot="tooltip">Export</svelte:fragment>
</Tooltip>

<Tooltip disabled={isRefreshing || !data.rows?.total} placement="top">
Expand Down
256 changes: 206 additions & 50 deletions ...roject-[region]-[project]/databases/database-[database]/table-[table]/export/+page.svelte
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,17 @@
import { table } from '../store';
import { queries, type TagValue } from '$lib/components/filters/store';
import { TagList } from '$lib/components/filters';
import { Submit, trackEvent, trackError } from '$lib/actions/analytics';
import { Click, Submit, trackEvent, trackError } from '$lib/actions/analytics';
import { toLocalDateTimeISO } from '$lib/helpers/date';
import { writable } from 'svelte/store';
import { isSmallViewport } from '$lib/stores/viewport';
import { Query } from '@appwrite.io/console';

let showExitModal = $state(false);
let formComponent: Form;
let isSubmitting = $state(writable(false));
let abortController: AbortController | null = null;
let exportProgress = $state(0);

let localQueries = $state<Map<TagValue, string>>(new Map());
const localTags = $derived(Array.from(localQueries.keys()));
Expand All @@ -29,7 +32,9 @@
.split('T')
.join('_')
.slice(0, -4);
const filename = `${$table.name}_${timestamp}.csv`;

let exportFormat = $state<'csv' | 'json'>('csv');
let filename = $derived(`${$table.name}_${timestamp}.${exportFormat}`);
Comment thread
Divyansh2992 marked this conversation as resolved.

let selectedColumns = $state<Record<string, boolean>>({});
let showAllColumns = $state(false);
Expand Down Expand Up @@ -97,34 +102,150 @@
return;
}

try {
await sdk
.forProject(page.params.region, page.params.project)
.migrations.createCSVExport({
resourceId: `${page.params.database}:${page.params.table}`,
filename: filename,
columns: selectedCols,
queries: exportWithFilters ? Array.from(localQueries.values()) : [],
delimiter: delimiterMap[delimiter],
header: includeHeader,
notify: true
if (exportFormat === 'csv') {
try {
await sdk
.forProject(page.params.region, page.params.project)
.migrations.createCSVExport({
resourceId: `${page.params.database}:${page.params.table}`,
filename: filename,
columns: selectedCols,
queries: exportWithFilters ? Array.from(localQueries.values()) : [],
delimiter: delimiterMap[delimiter],
header: includeHeader,
notify: true
});

addNotification({
type: 'success',
message: 'CSV export has started'
});

addNotification({
type: 'success',
message: 'CSV export has started'
});
trackEvent(Submit.DatabaseExportCsv);
await goto(tableUrl);
} catch (error) {
addNotification({
type: 'error',
message: error?.message || String(error)
});

trackEvent(Submit.DatabaseExportCsv);
trackError(error, Submit.DatabaseExportCsv);
}
} else {
$isSubmitting = true;
abortController = new AbortController(); // Initialize abort controller
exportProgress = 0; // Reset progress

await goto(tableUrl);
} catch (error) {
addNotification({
type: 'error',
message: error.message
});
try {
const activeQueries = exportWithFilters ? Array.from(localQueries.values()) : [];
const allRows: Record<string, unknown>[] = [];
Comment thread
Divyansh2992 marked this conversation as resolved.
const pageSize = 100;
let lastId: string | undefined = undefined;
let fetched = 0;
let total = Infinity;
let totalKnown = false;

while (fetched < total) {

const pageQueries = [Query.limit(pageSize), ...activeQueries];

if (lastId) {
pageQueries.push(Query.cursorAfter(lastId));
}
Comment thread
greptile-apps[bot] marked this conversation as resolved.
Outdated

const response = await sdk
.forProject(page.params.region, page.params.project)
.tablesDB.listRows({
databaseId: page.params.database,
tableId: page.params.table,
queries: pageQueries,
signal: abortController.signal
});

total = response.total;

if (response.rows.length === 0) break;
Comment thread
greptile-apps[bot] marked this conversation as resolved.

// After first page, we know the real total — notify the user
if (!totalKnown) {
totalKnown = true;
addNotification({
type: 'info',
message: `Exporting ${total.toLocaleString()} row${total !== 1 ? 's' : ''}…`,
timeout: 5000
});
if (total > 10_000) {
addNotification({
type: 'warning',
message: `Large export (${total.toLocaleString()} rows) — this may use significant browser memory.`
});
}
}

const filtered = response.rows.map((row) => {
const obj: Record<string, unknown> = {};
for (const col of selectedCols) {
obj[col] = row[col];
}
return obj;
});

allRows.push(...filtered);
fetched += response.rows.length;
lastId = response.rows[response.rows.length - 1].$id as string;
exportProgress = Math.min(100, Math.floor((fetched / total) * 100)); // Update progress
}

trackError(error, Submit.DatabaseExportCsv);
if (!abortController.signal.aborted) {
const json = JSON.stringify(allRows, null, 2);
const blob = new Blob([json], { type: 'application/json' });
const url = URL.createObjectURL(blob);
const anchor = document.createElement('a');
anchor.href = url;
anchor.download = filename;
document.body.appendChild(anchor);
anchor.click();

// Revoke the object URL after a short delay to ensure the browser has started the download
setTimeout(() => {
URL.revokeObjectURL(url);
document.body.removeChild(anchor);
}, 100);

addNotification({
type: 'success',
message: `JSON export complete — ${allRows.length} row${allRows.length !== 1 ? 's' : ''} downloaded`
});

trackEvent(Submit.DatabaseExportJson);

await goto(tableUrl);
}
} catch (error) {
if (error?.name === 'AbortError') {
addNotification({
type: 'warning',
message: 'JSON export cancelled.'
});
} else {
addNotification({
type: 'error',
message: error?.message || String(error)
});
trackError(error, Submit.DatabaseExportJson);
}
} finally {
$isSubmitting = false;
exportProgress = 0; // Reset progress
abortController = null; // Clean up controller
}
}
}

// Cancel the JSON export operation
function cancelExport() {
if (abortController) {
abortController.abort();
}
Comment thread
Divyansh2992 marked this conversation as resolved.
}

Expand All @@ -134,8 +255,21 @@
});
</script>

<Wizard title="Export CSV" columnSize="s" href={tableUrl} bind:showExitModal confirmExit column>
<Wizard title="Export" columnSize="s" href={tableUrl} bind:showExitModal confirmExit column>
<Form bind:this={formComponent} bind:isSubmitting onSubmit={handleExport}>
{#if exportFormat === 'json' && $isSubmitting}
<div class="progress-container" style="margin-top:1rem; display:flex; align-items:center; gap:0.5rem;">
<div
role="progressbar"
aria-label="Export progress"
aria-valuenow={exportProgress}
aria-valuemin="0"
aria-valuemax="100"
style="flex:1; background:linear-gradient(to right, #4caf50 {exportProgress}%, #e0e0e0 0%); height:0.5rem; border-radius:0.25rem;">
Comment thread
Divyansh2992 marked this conversation as resolved.
Outdated
</div>
<Button secondary compact on:click={cancelExport}>Cancel</Button>
</div>
{/if}
<Layout.Stack gap="xxl">
<Fieldset legend="Columns">
<Layout.Stack gap="l">
Expand Down Expand Up @@ -172,30 +306,45 @@
<Fieldset legend="Export options">
<Layout.Stack gap="l">
<InputSelect
id="delimiter"
label="Delimiter"
bind:value={delimiter}
id="exportFormat"
label="Format"
bind:value={exportFormat}
options={[
{ value: 'Comma', label: 'Comma' },
{ value: 'Semicolon', label: 'Semicolon' },
{ value: 'Tab', label: 'Tab' },
{ value: 'Pipe', label: 'Pipe' }
]}>
<Layout.Stack direction="row" gap="none" alignItems="center" slot="info">
<Tooltip>
<Icon size="s" icon={IconInfo} />
<span slot="tooltip">
Define how to separate values in the exported file.
</span>
</Tooltip>
</Layout.Stack>
</InputSelect>

<InputCheckbox
id="includeHeader"
label="Include header row"
description="Column names will be added as the first row in the CSV"
bind:checked={includeHeader} />
{ value: 'csv', label: 'CSV' },
{ value: 'json', label: 'JSON' }
]} />

{#if exportFormat === 'csv'}
<InputSelect
id="delimiter"
label="Delimiter"
bind:value={delimiter}
options={[
{ value: 'Comma', label: 'Comma' },
{ value: 'Semicolon', label: 'Semicolon' },
{ value: 'Tab', label: 'Tab' },
{ value: 'Pipe', label: 'Pipe' }
]}>
<Layout.Stack
direction="row"
gap="none"
alignItems="center"
slot="info">
<Tooltip>
<Icon size="s" icon={IconInfo} />
<span slot="tooltip">
Define how to separate values in the exported file.
</span>
</Tooltip>
</Layout.Stack>
</InputSelect>

<InputCheckbox
id="includeHeader"
label="Include header row"
description="Column names will be added as the first row in the CSV"
bind:checked={includeHeader} />
{/if}

<Layout.Stack gap="m">
<div class:disabled-checkbox={localTags.length === 0}>
Expand Down Expand Up @@ -233,7 +382,12 @@
</Button>
<Button
fullWidthMobile
on:click={() => formComponent.triggerSubmit()}
on:click={() => {
trackEvent(
exportFormat === 'json' ? Click.DatabaseExportJson : Click.DatabaseExportCsv
);
formComponent.triggerSubmit();
}}
disabled={$isSubmitting || selectedColumnCount === 0}>
Export
</Button>
Expand All @@ -245,4 +399,6 @@
.disabled-checkbox :global(*) {
cursor: unset;
}


</style>