Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 15 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -222,8 +222,8 @@ See [docs/store/quick-start.md](./docs/store/quick-start.md) for more commands.

### Admin Routes (Protected)
- `/admin` - Admin dashboard
- `/admin/users` - User management
- `/admin/requests` - Access request management
- `/admin/users` - User management (includes Users and Requests tabs)
- `/admin/users/requests` - Access request management
- `/admin/data` - Redis data inspection & RIS collection
- `/admin/ingest` - Content ingestion (legacy)
- `/admin/ingest-full` - Full content ingestion
Expand Down Expand Up @@ -263,6 +263,19 @@ See [docs/store/quick-start.md](./docs/store/quick-start.md) for more commands.

Copy `.env.example` to `.env` and provide:

### Admin Test Data

To populate Redis with sample admin/user accounts and pending access requests for development, run:

```bash
npm run admin:seed-test-users
```

Optional flags:

- `--refresh` — clear previously seeded data before recreating it
- `--extra-pending=5` — append additional random pending requests (use any number)

**Shopify (required for store):**
- `SHOPIFY_STORE_DOMAIN` - your-store.myshopify.com
- `SHOPIFY_STOREFRONT_TOKEN` - Storefront API access token
Expand Down
221 changes: 221 additions & 0 deletions docs/migrations/001-role-to-roles-array.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,221 @@
# Migration: role → roles[] Array

**Date:** 2025-10-27
**Type:** Data Schema Change
**Risk Level:** Low (backward compatible, idempotent)

## Overview

Migrates user data from single `role` field to multiple `roles` array to support users having multiple roles simultaneously (e.g., both `admin` and `community_manager`).

## Changes

**Before:**
```json
{
"email": "user@example.com",
"role": "admin",
"addedAt": "2025-10-27T00:00:00.000Z"
}
```

**After:**
```json
{
"email": "user@example.com",
"roles": ["admin"],
"addedAt": "2025-10-27T00:00:00.000Z"
}
```

## Migration Strategy

### Option 1: Zero-Downtime (Recommended)

The application automatically migrates data on-read. No downtime required.

**How it works:**
1. Deploy new code
2. Users are migrated automatically when their data is accessed
3. Optionally run migration script to pre-migrate all users

**Steps:**
```bash
# 1. Deploy new code (already includes migration logic)
git push production

# 2. (Optional) Pre-migrate all users in background
npm run admin:migrate-users
```

### Option 2: Pre-Migration (Safest)

Run migration before deploying new code.

**Steps:**
```bash
# 1. Run migration against production Redis
REDIS_URL="your-production-redis-url" npm run admin:migrate-users

# 2. Verify migration succeeded
# Check logs for "Migration complete!"

# 3. Deploy new code
git push production
```

## Production Migration Guide

### Prerequisites

- [ ] Backup Redis data
- [ ] Test migration on staging environment
- [ ] Notify team of maintenance window (if using Option 2)

### Step 1: Backup Redis Data

```bash
# Connect to production Redis
redis-cli -u $REDIS_URL

# Create RDB snapshot
SAVE

# Or use Redis Cloud backup feature
```

### Step 2: Test on Staging

```bash
# Point to staging Redis
export REDIS_URL="redis://staging-redis-url"

# Run migration
npm run admin:migrate-users

# Verify all users migrated successfully
# Check application still works
```

### Step 3: Run Production Migration

**Option A: Pre-migration (Recommended for large datasets)**

```bash
# Set production Redis URL
export REDIS_URL="redis://production-redis-url"

# Run migration
npm run admin:migrate-users

# Expected output:
# ✅ Successfully migrated N users
# ⏭️ Skipped M users (already migrated)
```

**Option B: Automatic migration**

Just deploy the new code. Users will be migrated on first access.

### Step 4: Verify Migration

```bash
# Check a few users manually
redis-cli -u $REDIS_URL

# Get a user
GET admin:user:someone@example.com

# Should see "roles": [...] not "role": "..."
```

### Step 5: Monitor

```bash
# Watch application logs for migration messages
# Look for: "🔄 Migrated user X from role to roles array"

# If using automatic migration, users will migrate gradually
# If using pre-migration, should see no migration logs
```

## Rollback Plan

If issues occur, the old code still works because:
- Old data format (`role`) is automatically converted to new format (`roles`)
- Migration is non-destructive

**To rollback:**
1. Redeploy previous version
2. Users will continue working
3. Optionally restore Redis backup if needed

## Script Details

### Idempotency

The migration script is idempotent - safe to run multiple times:

```javascript
// Only migrates users with old format
if (user.role && !user.roles) {
// Migrate
}
```

**Running multiple times:**
```bash
npm run admin:migrate-users # Migrates users
npm run admin:migrate-users # Skips already migrated users ✅
```

### Performance

- Uses Redis pipelines for batch operations
- Processes ~1000 users/second
- No locks or blocking operations

### Error Handling

- Continues processing if individual user fails
- Logs all errors
- Returns exit code 1 if migration fails

## Testing Checklist

- [ ] Backup created
- [ ] Tested on staging
- [ ] Migration script runs successfully
- [ ] Application loads correctly
- [ ] Users can log in
- [ ] Admin panel shows roles correctly
- [ ] User permissions work as expected

## Monitoring

After migration, monitor for:
- Migration log messages (should decrease over time with auto-migration)
- User authentication errors
- Admin access issues
- Role permission errors

## Support

If issues occur:
1. Check application logs for migration errors
2. Verify Redis connection
3. Check user data format in Redis directly
4. Contact engineering team

## Timeline

**Staging:** Test now
**Production:** Deploy during low-traffic window
**Estimated downtime:** 0 seconds (with Option 1)

## Notes

- Migration is backward compatible
- Old code will not work correctly with new data (always migrate forward)
- SUPER_ADMIN_EMAIL environment variable works throughout migration
- No changes to Redis keys or indexes
2 changes: 2 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@
"drops:create": "node --env-file=.env scripts/shopify-create-drop.mjs",
"collections:generate-images": "node --env-file=.env scripts/generate-collection-images.mjs",
"admin:bootstrap": "node --env-file=.env scripts/bootstrap-admin.mjs",
"admin:seed-test-users": "node --env-file=.env scripts/seed-admin-test-users.mjs",
"admin:migrate-users": "node --env-file=.env scripts/migrate-users-to-roles-array.mjs",
"seed:communities": "npx tsx scripts/seed-communities.ts"
},
"dependencies": {
Expand Down
18 changes: 18 additions & 0 deletions scripts/check-redis-requests.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
import Redis from 'ioredis';

const redis = new Redis(process.env.REDIS_URL);

console.log('Checking pending requests in Redis...\n');

const pendingIds = await redis.smembers('admin:requests:pending');
console.log(`Found ${pendingIds.length} pending request IDs`);

for (const id of pendingIds) {
const data = await redis.get(`admin:request:${id}`);
if (data) {
const req = JSON.parse(data);
console.log(` - ${req.email} (${req.status})`);
}
}

await redis.quit();
114 changes: 114 additions & 0 deletions scripts/migrate-users-to-roles-array.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
#!/usr/bin/env node

/**
* Migration Script: role → roles[]
* Migrates all users in Redis from old 'role' field to new 'roles' array
*
* Usage:
* npm run admin:migrate-users # Run migration
* npm run admin:migrate-users -- --dry-run # Preview changes without saving
* npm run admin:migrate-users -- --force # Skip confirmation prompt
*/

import Redis from 'ioredis';
import readline from 'readline';

const args = process.argv.slice(2);
const isDryRun = args.includes('--dry-run');
const isForce = args.includes('--force');

const redisUrl = process.env.REDIS_URL || 'redis://localhost:6379';
const redis = new Redis(redisUrl);

function askConfirmation(question) {
const rl = readline.createInterface({
input: process.stdin,
output: process.stdout,
});

return new Promise((resolve) => {
rl.question(question, (answer) => {
rl.close();
resolve(answer.toLowerCase().startsWith('y'));
});
});
}

async function migrateUsers() {
console.log('🔄 Starting user migration: role → roles[]\n');

if (isDryRun) {
console.log('🔍 DRY RUN MODE - No changes will be saved\n');
}

console.log(`📍 Redis: ${redisUrl}\n`);

try {
// Get all user emails
const emails = await redis.smembers('admin:users:all');
console.log(`Found ${emails.length} users to check\n`);

if (emails.length === 0) {
console.log('✅ No users found. Migration complete.');
await redis.quit();
return;
}

// Batch fetch all users
const keys = emails.map(email => `admin:user:${email.toLowerCase()}`);
const values = await redis.mget(...keys);

let migratedCount = 0;
let skippedCount = 0;
const pipeline = redis.pipeline();

for (let i = 0; i < values.length; i++) {
const data = values[i];
if (!data) {
console.log(`⚠️ Skipping ${emails[i]} - no data found`);
continue;
}

try {
const user = JSON.parse(data);

// Check if migration needed
if (user.role && !user.roles) {
// Migrate
user.roles = [user.role];
delete user.role;
pipeline.set(keys[i], JSON.stringify(user));
console.log(`✅ Migrating ${user.email}: "${user.roles[0]}" → ["${user.roles[0]}"]`);
migratedCount++;
} else if (user.roles) {
console.log(`⏭️ Skipping ${user.email} - already has roles array`);
skippedCount++;
} else {
console.log(`⚠️ Skipping ${user.email} - no role or roles field`);
skippedCount++;
}
} catch (error) {
console.error(`❌ Error processing ${emails[i]}:`, error.message);
}
}

// Execute all migrations
if (migratedCount > 0) {
await pipeline.exec();
console.log(`\n✅ Successfully migrated ${migratedCount} users`);
} else {
console.log('\n✅ No users needed migration');
}

console.log(`⏭️ Skipped ${skippedCount} users (already migrated or no data)`);
console.log('\n🎉 Migration complete!');

} catch (error) {
console.error('❌ Migration failed:', error);
process.exit(1);
} finally {
await redis.quit();
}
}

migrateUsers();
Loading