Data Fetching Strategies

3. Data Fetching Strategies

One of Next.js’s most powerful features is its flexible and optimized data fetching mechanisms. Depending on your application’s needs, you can choose from various strategies, each with its own benefits regarding performance, SEO, and user experience. This chapter will cover the primary ways to fetch data in Next.js, especially with the App Router.

3.1 Fetching Data in Server Components

As we learned, Server Components run on the server and are the default in the App Router. This means they can directly access server-side resources like databases or internal APIs without exposing sensitive credentials to the client.

Using fetch() API

The native fetch() API is automatically extended by Next.js to provide powerful caching and revalidation capabilities. You can use async/await directly in your Server Components.

Example: Fetching a list of posts from an external API

Let’s imagine we want to fetch a list of blog posts to display on a new /blog route.

  1. Create a new route segment src/app/blog.

    mkdir src/app/blog
    touch src/app/blog/page.tsx
    
  2. Add the following code to src/app/blog/page.tsx:

    // src/app/blog/page.tsx
    
    // Define a type for a Post
    interface Post {
      id: number;
      title: string;
      body: string;
    }
    
    // This component is a Server Component by default
    export default async function BlogPage() {
      // Simulate a network delay (optional, for demonstration)
      // await new Promise(resolve => setTimeout(resolve, 2000));
    
      const response = await fetch('https://jsonplaceholder.typicode.com/posts?_limit=10', {
        // Next.js extends fetch to allow caching options
        // 'force-cache': default, caches data until manually revalidated or deploy
        // 'no-store': always fetches fresh data
        // next: { revalidate: 60 }: revalidates data every 60 seconds
        cache: 'no-store' // This ensures fresh data on every request to this page
      });
      const posts: Post[] = await response.json();
    
      return (
        <main style={{ padding: '20px', fontFamily: 'sans-serif' }}>
          <h1>Latest Blog Posts</h1>
          <p>This data was fetched on the server!</p>
          <ul style={{ listStyle: 'none', padding: 0 }}>
            {posts.map((post) => (
              <li key={post.id} style={{ marginBottom: '15px', borderBottom: '1px solid #eee', paddingBottom: '10px' }}>
                <h3 style={{ color: '#0070f3', marginBottom: '5px' }}>{post.title}</h3>
                <p>{post.body.substring(0, 100)}...</p>
              </li>
            ))}
          </ul>
        </main>
      );
    }
    
  3. Save the file and navigate to http://localhost:3000/blog. You’ll see the blog posts. If you refresh the page multiple times, with cache: 'no-store', the data is fetched fresh on each full page load. If you change it to cache: 'force-cache' (or remove the cache option, as it’s the default), Next.js will cache the data and subsequent requests will serve the cached version.

Direct Database/ORM Queries

Since Server Components run on the server, you can directly query your database or use an ORM (Object-Relational Mapper) without needing an extra API layer. This simplifies your architecture and improves security by keeping database credentials strictly on the server.

Example: Simulating a direct database query

While we won’t set up a real database in this guide, we can simulate the pattern.

  1. Create a utility file src/lib/db.ts to simulate database interaction.

    mkdir src/lib
    touch src/lib/db.ts
    
  2. Add content to src/lib/db.ts:

    // src/lib/db.ts
    
    interface User {
      id: number;
      name: string;
      email: string;
    }
    
    const users: User[] = [
      { id: 1, name: 'Alice Smith', email: 'alice@example.com' },
      { id: 2, name: 'Bob Johnson', email: 'bob@example.com' },
      { id: 3, name: 'Charlie Brown', email: 'charlie@example.com' },
    ];
    
    export async function getUsersFromDb(): Promise<User[]> {
      // Simulate a database query delay
      await new Promise(resolve => setTimeout(resolve, 1500));
      console.log('Fetching users directly from (simulated) DB...');
      return users;
    }
    
    export async function getUserByIdFromDb(id: number): Promise<User | undefined> {
        await new Promise(resolve => setTimeout(resolve, 1000));
        console.log(`Fetching user ${id} directly from (simulated) DB...`);
        return users.find(user => user.id === id);
    }
    
  3. Create a new route for users: src/app/users/page.tsx.

    mkdir src/app/users
    touch src/app/users/page.tsx
    
  4. Add content to src/app/users/page.tsx:

    // src/app/users/page.tsx
    import { getUsersFromDb } from '@/lib/db'; // Import the simulated database function
    import Link from 'next/link';
    
    export default async function UsersPage() {
      // This Server Component directly calls a "database" function
      const users = await getUsersFromDb();
    
      return (
        <main style={{ padding: '20px', fontFamily: 'sans-serif' }}>
          <h1>User List</h1>
          <p>This data is fetched directly from a (simulated) database on the server.</p>
          <ul style={{ listStyle: 'disc', paddingLeft: '20px' }}>
            {users.map((user) => (
              <li key={user.id} style={{ marginBottom: '10px' }}>
                <Link href={`/users/${user.id}`} style={{ textDecoration: 'none', color: '#0070f3' }}>
                  {user.name} ({user.email})
                </Link>
              </li>
            ))}
          </ul>
        </main>
      );
    }
    
  5. Create a dynamic route for a single user: src/app/users/[userId]/page.tsx.

    mkdir src/app/users/[userId]
    touch src/app/users/[userId]/page.tsx
    
  6. Add content to src/app/users/[userId]/page.tsx:

    // src/app/users/[userId]/page.tsx
    import { getUserByIdFromDb } from '@/lib/db';
    import { notFound } from 'next/navigation'; // Next.js utility for 404 pages
    import Link from 'next/link';
    
    interface UserDetailPageProps {
      params: {
        userId: string;
      };
    }
    
    export default async function UserDetailPage({ params }: UserDetailPageProps) {
      const id = parseInt(params.userId);
    
      // Fetch user from simulated DB directly in the Server Component
      const user = await getUserByIdFromDb(id);
    
      if (!user) {
        // If user is not found, render Next.js's default 404 page
        notFound();
      }
    
      return (
        <main style={{ padding: '20px', fontFamily: 'sans-serif' }}>
          <h2>User Details: {user.name}</h2>
          <p>Email: {user.email}</p>
          <p>ID: {user.id}</p>
          <p>
            <Link href="/users" style={{ textDecoration: 'none', color: '#0070f3' }}>
              &larr; Back to User List
            </Link>
          </p>
        </main>
      );
    }
    
  7. Save all files. Navigate to http://localhost:3000/users and then click on individual users. You’ll observe the data being fetched directly on the server. Look at your terminal where npm run dev is running; you should see the console.log messages for database fetches. This demonstrates that the code runs entirely on the server.

Exercises/Mini-Challenges (Server Component Data Fetching):

  1. Implement Revalidation for Blog Posts:

    • In src/app/blog/page.tsx, change the fetch options to revalidate data every 10 seconds.
    • Hint: next: { revalidate: 10 }
    • How would you test this without a constantly changing API? (Hint: Imagine the API data would change, and notice that Next.js automatically caches and revalidates in the background.)
  2. Display a “No Posts Found” Message:

    • Modify src/app/blog/page.tsx so that if the posts array is empty, it renders a message like “No blog posts available at the moment.” instead of an empty list.

3.2 Streaming and Loading UI with loading.tsx and Suspense

When data fetching takes time, you don’t want your users to stare at a blank screen. Next.js, in conjunction with React’s Suspense, allows you to stream parts of your UI as they become ready.

loading.tsx (for route segments)

You can create a loading.tsx file inside any route segment (folder) to automatically display a loading state for that segment and its children while their data is being fetched.

Example: Adding a loading UI for the blog posts

  1. Create a loading.tsx file inside src/app/blog.

    touch src/app/blog/loading.tsx
    
  2. Add the following content to src/app/blog/loading.tsx:

    // src/app/blog/loading.tsx
    
    export default function Loading() {
      // You can return any UI here, e.g., a spinner or skeleton screen
      return (
        <div style={{
          padding: '20px',
          fontFamily: 'sans-serif',
          textAlign: 'center',
          color: '#888'
        }}>
          <h2 style={{ color: '#0070f3' }}>Loading Blog Posts...</h2>
          <div style={{
            border: '4px solid rgba(0, 0, 0, .1)',
            width: '36px',
            height: '36px',
            borderRadius: '50%',
            borderLeftColor: '#0070f3',
            animation: 'spin 1s ease infinite',
            margin: '20px auto'
          }}></div>
          <p>Please wait while we fetch the latest articles.</p>
    
          <style jsx>{`
            @keyframes spin {
              0% { transform: rotate(0deg); }
              100% { transform: rotate(360deg); }
            }
          `}</style>
        </div>
      );
    }
    
  3. Re-introduce a delay in src/app/blog/page.tsx to clearly see the loading state. Uncomment the await new Promise(...) line:

    // src/app/blog/page.tsx
    // ...
    export default async function BlogPage() {
      await new Promise(resolve => setTimeout(resolve, 2000)); // Simulate delay
      // ...
    }
    
  4. Save both files. Navigate to http://localhost:3000/blog. When you navigate to this page, you should briefly see the “Loading Blog Posts…” UI before the actual posts appear. Next.js automatically wraps your page.tsx (and any nested layout.tsx) with this loading.tsx inside a Suspense boundary.

React Suspense (for granular control)

While loading.tsx is great for whole route segments, Suspense gives you more granular control. You can wrap any component that fetches data (or is a Server Component waiting for data) with <Suspense fallback={<LoadingSpinner />} /> to show a loading state specifically for that component.

This allows other parts of the page that don’t depend on that data to render immediately, improving perceived performance.

Example: Granular loading for a slow component

Let’s say our list of users (/users) loads quickly, but we want to add another component that fetches additional, slower data on the same page.

  1. Create a slow Server Component: src/app/users/components/SlowUserStats.tsx.

    touch src/app/users/components/SlowUserStats.tsx
    
  2. Add content to src/app/users/components/SlowUserStats.tsx:

    // src/app/users/components/SlowUserStats.tsx
    
    interface UserStat {
      id: number;
      label: string;
      value: number;
    }
    
    async function fetchUserStats(): Promise<UserStat[]> {
      // Simulate a *very* slow data fetch
      await new Promise(resolve => setTimeout(resolve, 4000));
      console.log('Fetching slow user stats...');
      return [
        { id: 1, label: 'Total Active Users', value: 12345 },
        { id: 2, label: 'New Signups Today', value: 87 },
        { id: 3, label: 'Churn Rate', value: 2.5 },
      ];
    }
    
    export default async function SlowUserStats() {
      const stats = await fetchUserStats();
    
      return (
        <div style={{
          marginTop: '30px',
          border: '1px solid #ccc',
          padding: '15px',
          borderRadius: '8px',
          backgroundColor: '#f0f8ff'
        }}>
          <h4>User Statistics (Slow Loading)</h4>
          <ul style={{ listStyle: 'none', padding: 0 }}>
            {stats.map(stat => (
              <li key={stat.id} style={{ marginBottom: '8px' }}>
                <strong>{stat.label}:</strong> {stat.value}
              </li>
            ))}
          </ul>
        </div>
      );
    }
    
  3. Create a loading fallback component: src/app/users/components/UserStatsLoading.tsx.

    touch src/app/users/components/UserStatsLoading.tsx
    
  4. Add content to src/app/users/components/UserStatsLoading.tsx:

    // src/app/users/components/UserStatsLoading.tsx
    
    export default function UserStatsLoading() {
      return (
        <div style={{
          marginTop: '30px',
          border: '1px dashed #aaddff',
          padding: '15px',
          borderRadius: '8px',
          backgroundColor: '#e6f7ff',
          color: '#0070f3',
          textAlign: 'center'
        }}>
          <p>Loading user statistics...</p>
          {/* Simple skeleton loader */}
          <div style={{ height: '20px', backgroundColor: '#bbddff', borderRadius: '4px', margin: '10px 0', width: '80%' }}></div>
          <div style={{ height: '20px', backgroundColor: '#bbddff', borderRadius: '4px', margin: '10px 0', width: '60%' }}></div>
        </div>
      );
    }
    
  5. Use Suspense in src/app/users/page.tsx:

    // src/app/users/page.tsx
    import { getUsersFromDb } from '@/lib/db';
    import Link from 'next/link';
    import { Suspense } from 'react'; // Import Suspense
    import SlowUserStats from './components/SlowUserStats'; // Import the slow component
    import UserStatsLoading from './components/UserStatsLoading'; // Import its loading fallback
    
    export default async function UsersPage() {
      const users = await getUsersFromDb(); // This fetch might be fast
    
      return (
        <main style={{ padding: '20px', fontFamily: 'sans-serif' }}>
          <h1>User List</h1>
          <p>This data is fetched directly from a (simulated) database on the server.</p>
          <ul style={{ listStyle: 'disc', paddingLeft: '20px' }}>
            {users.map((user) => (
              <li key={user.id} style={{ marginBottom: '10px' }}>
                <Link href={`/users/${user.id}`} style={{ textDecoration: 'none', color: '#0070f3' }}>
                  {user.name} ({user.email})
                </Link>
              </li>
            ))}
          </ul>
    
          {/* Wrap the slow component with Suspense */}
          <Suspense fallback={<UserStatsLoading />}>
            <SlowUserStats />
          </Suspense>
        </main>
      );
    }
    
  6. Save all files. Navigate to http://localhost:3000/users. You’ll see the user list immediately, and then after a 4-second delay, the “User Statistics” section will appear. This demonstrates how Suspense allows your UI to progressively load, showing meaningful content as soon as possible.

Exercises/Mini-Challenges (Streaming & Suspense):

  1. Customize loading.tsx for Users:

    • Create a loading.tsx file inside src/app/users similar to the one for blog.
    • Observe how this interacts with the Suspense boundary on SlowUserStats. Does the root loading.tsx or the Suspense fallback take precedence for the SlowUserStats component’s loading state? (Hint: The innermost Suspense boundary’s fallback takes precedence for its direct children.)
  2. Add another slow component:

    • Create src/app/users/components/EvenSlowerFeature.tsx that simulates an 8-second delay.
    • Create a corresponding EvenSlowerFeatureLoading.tsx.
    • Integrate EvenSlowerFeature into src/app/users/page.tsx using its own Suspense boundary. Notice how multiple slow components can load independently.

3.3 Client-Side Data Fetching

While Server Components are the preferred method for initial data loads, there are scenarios where you need to fetch data client-side after the initial page render. This is common for:

  • Data that frequently changes and doesn’t require SEO.
  • Data that is specific to user interaction (e.g., clicking a button to load more results).
  • Data that depends on browser-specific APIs.
  • Integrating with third-party libraries designed for client-side data fetching (like SWR or React Query).

Using useEffect and useState (basic approach)

For simple client-side fetches, you can use React’s useEffect and useState hooks within a Client Component.

Example: Client-side joke fetching

Let’s create a page that fetches a new joke every time a button is clicked.

  1. Create a new route src/app/jokes.

    mkdir src/app/jokes
    touch src/app/jokes/page.tsx
    
  2. Add content to src/app/jokes/page.tsx:

    // src/app/jokes/page.tsx
    'use client'; // This is a Client Component
    
    import { useState, useEffect } from 'react';
    import Link from 'next/link';
    
    interface Joke {
      id: string;
      joke: string;
      status: number;
    }
    
    export default function JokesPage() {
      const [joke, setJoke] = useState<string | null>(null);
      const [loading, setLoading] = useState(false);
      const [error, setError] = useState<string | null>(null);
    
      const fetchJoke = async () => {
        setLoading(true);
        setError(null);
        setJoke(null); // Clear previous joke
    
        try {
          const response = await fetch('https://icanhazdadjoke.com/', {
            headers: {
              'Accept': 'application/json',
            },
          });
    
          if (!response.ok) {
            throw new Error(`HTTP error! status: ${response.status}`);
          }
    
          const data: Joke = await response.json();
          setJoke(data.joke);
        } catch (err: any) {
          setError(`Failed to fetch joke: ${err.message}`);
          console.error('Error fetching joke:', err);
        } finally {
          setLoading(false);
        }
      };
    
      // Fetch a joke when the component mounts
      useEffect(() => {
        fetchJoke();
      }, []); // Empty dependency array means it runs once on mount
    
      return (
        <main style={{ padding: '20px', fontFamily: 'sans-serif' }}>
          <h1>Dad Jokes (Client-Side)</h1>
          <p>This joke is fetched dynamically on the client after the page loads.</p>
    
          <div style={{
            border: '1px solid #ddd',
            padding: '20px',
            borderRadius: '8px',
            backgroundColor: '#fff',
            maxWidth: '600px',
            margin: '20px 0'
          }}>
            {loading && <p style={{ color: '#0070f3' }}>Loading a new joke...</p>}
            {error && <p style={{ color: 'red' }}>Error: {error}</p>}
            {joke && <p style={{ fontSize: '1.2em', fontStyle: 'italic' }}>"{joke}"</p>}
          </div>
    
          <button
            onClick={fetchJoke}
            disabled={loading}
            style={{
              padding: '10px 20px',
              backgroundColor: '#0070f3',
              color: 'white',
              border: 'none',
              borderRadius: '5px',
              cursor: loading ? 'not-allowed' : 'pointer'
            }}
          >
            {loading ? 'Fetching...' : 'Get Another Joke'}
          </button>
    
          <p style={{ marginTop: '20px' }}>
            <Link href="/" style={{ textDecoration: 'none', color: '#0070f3' }}>
              &larr; Back to Home
            </Link>
          </p>
        </main>
      );
    }
    
  3. Save the file and navigate to http://localhost:3000/jokes. You’ll see a joke and can click the button to fetch new ones. The network requests will happen in your browser’s developer tools, confirming client-side fetching.

Using SWR or React Query (for advanced scenarios)

For more complex client-side data fetching needs, especially with caching, revalidation, and state management, libraries like SWR (developed by Vercel) and React Query are excellent choices. They provide powerful hooks that abstract away much of the boilerplate associated with data fetching, offering features like automatic re-fetching, caching, and error handling.

We won’t go into a full setup here as it involves installing external libraries and setting up a SWRConfig or QueryClientProvider, but it’s important to know they exist and are highly recommended for applications with significant client-side data fetching.

Conceptual Example (SWR):

// src/app/dashboard/components/ActivityFeed.tsx
'use client';

import useSWR from 'swr'; // You'd need to `npm install swr`
import { Suspense } from 'react';

interface Activity {
  id: number;
  message: string;
  timestamp: string;
}

const fetcher = (url: string) => fetch(url).then(res => res.json());

export default function ActivityFeed() {
  const { data, error, isLoading } = useSWR<Activity[]>('/api/user/activity', fetcher, {
    // SWR options
    revalidateOnFocus: true, // Revalidate when window regains focus
    refreshInterval: 5000,   // Revalidate every 5 seconds
  });

  if (error) return <p style={{ color: 'red' }}>Failed to load activity feed.</p>;
  if (isLoading) return <p style={{ color: '#0070f3' }}>Loading recent activity...</p>;
  if (!data || data.length === 0) return <p>No activity yet.</p>;

  return (
    <div style={{
      marginTop: '20px',
      border: '1px solid #eee',
      padding: '15px',
      borderRadius: '8px',
      backgroundColor: '#fdfdfd'
    }}>
      <h3>Recent User Activity</h3>
      <ul style={{ listStyle: 'none', padding: 0 }}>
        {data.map(activity => (
          <li key={activity.id} style={{ marginBottom: '5px' }}>
            <small>({new Date(activity.timestamp).toLocaleTimeString()}):</small> {activity.message}
          </li>
        ))}
      </ul>
    </div>
  );
}

// To use this, you'd integrate it into a page,
// potentially by creating a new `src/app/api/user/activity/route.ts`
// that returns mock activity data.

Exercises/Mini-Challenges (Client-Side Data Fetching):

  1. Add a Loading Indicator to JokesPage Button:

    • The button already has a disabled state based on loading.
    • Change the button’s text to “Loading…” when loading is true. (You’ve already done this, but reinforce the pattern.)
  2. Display Error State in JokesPage:

    • Introduce a network error (e.g., change the API URL to a non-existent one like https://not-a-real-api.com) in src/app/jokes/page.tsx.
    • Observe how your error message is displayed. Fix the URL back to the correct one after testing.
  3. Implement a Client-Side Timer:

    • Create a new client component src/app/components/ClientTimer.tsx.
    • This component should use useState and useEffect to display a continuously updating timer (e.g., seconds elapsed since component mount).
    • Integrate this ClientTimer into your src/app/page.tsx (the homepage). Since page.tsx is a Server Component, you’ll need to remember to make ClientTimer a 'use client' component.

By mastering these data fetching strategies, you’ll be well-equipped to build highly performant and user-friendly Next.js applications that efficiently handle data from various sources.