How we use gRPC to build our frontend at Polar Signals

We have written in the past about how we use gRPC to build our APIs here at Polar Signals. Now let us also see how we work with a gRPC backend API in the frontend.

gRPC is a modern, HTTP2-based protocol, that is based around the idea of defining a service, specifying the methods that can be called remotely with their parameters and return types.

gRPC uses protocol buffers as the Interface Definition Language (IDL) for describing both the service interface and the structure of the payload messages. The idea is that with a protocol buffer file (a .proto file) you can generate a server and a client for most programming languages.

However gRPC itself still isn’t fully supported in modern browsers due to gRPC being heavily dependent on HTTP/2, and no modern browser can access HTTP/2 frames. This is where gRPC-web comes in, as it serves as a proxy between the browser and gRPC backend.

In this article, we’ll take a look at how we use gRPC-web to build our frontend components and pages and work with API data coming from a gRPC server.

What is gRPC-web?

gRPC-web is a JavaScript implementation of gRPC for browser clients. It gives you all the advantages of working with gRPC, such as efficient serialization, a simple Interface Definition Language (IDL), and easy interface updating.

It’s not uncommon that a lot of APIs being built out there are built to follow the REST methodology or even the newer GraphQL. So why even choose gRPC at all? This is because of many reasons:

• gRPC messages can be up to 30 percent smaller in size than JSON messages. This is particularly important for us because our APIs tend to be data-heavy.
• No more hand-crafted JSON call objects – all requests and responses are strongly typed and code-generated, with hints available in the IDE.
• gRPC enables you to automatically generate code for multiple programming languages such as Go, Java, Python, and C++.
• No more second-guessing the meaning of error codes - gRPC status codes are a canonical way of representing issues in APIs.

Now that we’ve established why gRPC-web, let’s go into our frontend gRPC-web setup.

Firstly, our client code is generated by using the protobuf-ts package, which helps to generate client and server code if given a `proto.file`. Therefore, running protobuf-ts on our proto files will generate some client files like this: “@polarsignals/client/organization/organization.client”.

We use these client files to get the request methods and types needed to call services like a `GetOrganizations` or `GetUser` service.

To use these client files and work with the API, we had a few ideas on how to do data fetching, caching, and data mutation and we wanted to do them in the best possible way. So we came up with an architectural pattern that can be described in the sketch below.

Let’s walk through this sketch.

• It all begins from a `GRPCContext` file where we initialize the connection to the API and create a React context.
• The `useGRPCQuery` is where we utilize React Query, a data fetching library, to create a custom hook where we abstract out the code to handle data fetching, mutation, and caching.
• The “useAction” hook is representative of a set of custom hooks e.g. `useUser`, `useOrganizations`, that we set up to fetch and update some particular data.
• We can then use any of the “useAction” hooks (useUser, useOrganizations) in a React component.

We’ll now look at each of these pieces and how we implemented them.

Initializing with the GrpcContext file

The first thing we did was to create an overarching context file that contains our gRPC setup. This is important because it means every React component that is within the context gets to use the setup without needing to do the setup in multiple instances.

So for the GRPC Context, we expose a `GrpcContextProvider` provider and a `useGrpcContext` hook which are exported. Let’s look at what that code looks like.

// /contexts/GrpcContext/index.tsx
import React from 'react';

import {OrganizationsClient} from '@polarsignals/client-grpc-web/polarsignals/organization/organization.client';
import {ProjectsClient} from '@polarsignals/client-grpc-web/polarsignals/project/project.client';
import {UsersClient} from '@polarsignals/client-grpc-web/polarsignals/user/user.client';
import {GrpcWebFetchTransport} from '@protobuf-ts/grpcweb-transport';
import {apiEndpoint} from '../../constants';

const GRPC_SERVICE_HOST = `${apiEndpoint}/api`;

const GRPC_TRANSPORT = new GrpcWebFetchTransport({
  baseUrl: GRPC_SERVICE_HOST,
  fetchInit: {credentials: 'include'},
  format: 'binary',
});

interface IGrpcContext {
  usersClient: UsersClient;
  projectsClient: ProjectsClient;
  organizationsClient: OrganizationsClient;
}

const defaultValue: IGrpcContext = {
  usersClient: new UsersClient(GRPC_TRANSPORT),
  projectsClient: new ProjectsClient(GRPC_TRANSPORT),
  organizationsClient: new OrganizationsClient(GRPC_TRANSPORT),
};

const GrpcContext = React.createContext<IGrpcContext>(defaultValue);

export const GrpcContextProvider = ({children}: {children: React.ReactNode}) => {
  return <GrpcContext.Provider value={defaultValue}>{children}></GrpcContext.Provider>;
};

export const useGrpcContext = () => {
  const context = React.useContext(GrpcContext);
  if (context === undefined) {
    throw new Error('useGrpcContext must be used within a GrpcContextProvider');
  }

  return context;
};

export default GrpcContext;

In the file above, we first import the client files that were generated using `protobuf-ts`. These client files contain classes and objects that we can use to make requests to the API.

Then, we set up the connection to the API by initializing a new instance of `GrpcWebFetchTransport` and passing it the `baseUrl`, and `fetchInit` which is an options object to pass through to the fetch when doing a request. In this case, we set `credentials` to be `include` which will make requests including cookies for cross-origin calls. `GrpcWebFetchTransport` itself is imported from the protobuf-ts/grpcweb-transport package and is essential for communicating over gRPC-web.

Next, we went ahead and created the `GrpcContext` which has a default value of an object that contains all the clients we’ll use in our frontend application. Finally, we export a `GrpcContextProvider` context provider and also a `useGrpcContext` hook.

The `GrpcContextProvider` is used in the entry component of our application which is the `_app.tsx` file and placed at the top of the component tree. While the `useGrpcContext` hook is used in the useAction hooks. We’ll get to that further below in a bit.

Create a useGRPCQuery hook using React Query

For data fetching, we wanted a simple solution that also enabled automatically refetching of data and that’s why we went with React Query.

React Query is a library for data fetching, updating, and caching your remote data. It comes with an out-of-the-box mechanism to keep data fresh, by re-fetching data automatically when it is needed and defined as stale.

With the `useGRPCQuery` hook, we abstract data fetching to be in one single hook, which can then be utilized further in any of the useAction hooks.

We’ll utilize the useQuery hook from React Query to create the `useGRPCQuery hook`.

// /hooks/data/useGrpcQuery/index.ts
import {useQuery, UseQueryResult} from 'react-query';

interface Props<IRes> {
  key: string | any[];
  queryFn: () => Promise<IRes>;
  options?: {
    enabled?: boolean | undefined;
  };
}

const useGrpcQuery = <IRes>({
  key,
  queryFn,
  options: {enabled = true} = {},
}: Props<IRes>): UseQueryResult<IRes> => {

  return useQuery<IRes>(
    key,
    async () => {
      return await queryFn();
    },
    {
      enabled,
    }
  );
};

export default useGrpcQuery;

The `useGRPCQuery` hook essentially takes a `key` value, an asynchronous function for fetching data, and then returns various values that inform us about the current request. Let’s walk through the code.

The first property in the `useGrpcQuery` hook is a unique `key` and this is how the hook knows what to cache when data is returned. The unique key provided is used internally for refetching, caching, and sharing queries throughout the application.

The `queryFn` is the second parameter, it is an asynchronous function and is responsible for the actual fetch and this needs to return a promise. The `queryFn` can be the native Fetch API or a library like axios, as React Query itself doesn't necessarily care about the fetch API or any of the actual mechanisms or protocols that you're using. It only requires a promise to be returned.

Lastly, the `options` property holds the query’s configuration which in this case is the `enabled` value. A truthy `enabled` value means automatic refetching when the query mounts or the query key changes.

Now let’s see how the `useGRPCQuery` hook and GRPC context work together in the useAction hooks below.

useAction hooks

The useAction hooks are a set of custom hooks that we use to interact with the API in React components. It is responsible for fetching and updating data.

An individual hook is lightweight with just the following responsibilities:

• It gets the GRPC request object from the GRPC Context and then makes the appropriate request in the useGrpcQuery custom hook.
• It should return an object with the format: {loading, data, error}. Where data.<name> can contain the object, for example in the case of the user hook, data.user can have the requested data.

Here’s an example of one of the useAction hooks, `useOrganizations`. This action hook is responsible for fetching and updating details about an organization, as well as adding & removing users from an organization.

// /hooks/data/useOrganizations/index.ts
import {useCallback, useContext} from 'react';
import {Organization} from '@polarsignals/client-grpc-web/polarsignals/organization/organization';
import {User} from '@polarsignals/client-grpc-web/polarsignals/user/user';
import Toast from 'components/Toaster';
import GrpcContext from 'contexts/GrpcContext';
import useGrpcQuery from '../useGrpcQuery';

export const useOrganizationUsers = (id: string) => {
  const {organizationsClient} = useContext(GrpcContext);
  const {
    data: organizationUsers,
    isLoading,
    error,
    refetch,
  } = useGrpcQuery<User[]>({
    key: ['getOrganizationUsers', id],
    queryFn: async () => {
      const {response} = await organizationsClient.getOrganizationUsers({id: id});
      return response.users ?? [];
    },
  });

  const addOrganizationUser = (id: string, email: string) => {
    const call = organizationsClient.addOrganizationUser({id: id, email: email});
    call.response
      .then(() => {
        Toast('success', `${email} was added successfully`);
        refetch();
      })
      .catch(err => {
        console.error(err);
        Toast('error', `Error adding the new user, please try again: ${err.message}`);
      });
  };

  const deleteOrganizationUser = (id: string, userId: string) => {
    const call = organizationsClient.removeOrganizationUser({id: id, userId: userId});
    call.response
      .then(() => {
        Toast('success', 'User was removed successfully');
        refetch();
      })
      .catch(err => {
        console.error(err);
        Toast('error', `Error removing the user, please try again: ${err.message}`);
      });
  };

  return {
    data: {organizationUsers},
    loading: isLoading,
    error,
    mutations: {
      addOrganizationUser,
      deleteOrganizationUser,
    },
  };
};

export default useOrganizationUsers;

The custom hook above can be segmented into three parts; the first part of the code block uses the `useGrpcQuery` hook to fetch the organization’s users (this is the initial data returned by this hook), and the second part consists of multiple mutation functions that help to add/delete an organization’s users, and lastly, a returned object that contains the initial data, the loading & error state, and the mutations as an object.

We started by creating a function called `useOrganizationUsers` which is the name of this action hook. We get the `organizationsClient` client from the `GrpcContext` by destructuring the context object.

Next, we utilize the `useGrpcQuery` hook to fetch all the users in an organization. As you can see in the code, we are passing a unique key and an asynchronous function, and this hook returns some values. One of those values is `refetch`, refetch is a function returned from the custom `useGrpcQuery` hook (React Query) and it’s used to manually re-fetch the query. This helps make sure we have the latest data after a mutation and will re-render the UI with the new data.

The functions after the `useGrpcQuery` code block are mutation functions as they help us update data to the API, and just like the `useGrpcQuery`, these functions use the `organizationsClient` client to make requests to the API. If there’s a successful response, we show a success message and call the `refetch()` function so that the initial data is updated again.

Finally, at the end of the file, we return the initial data, the loading & error state, and the mutations defined above.

Usage of action hooks in Presentational Components

To demonstrate how the hook defined above is used, here’s a snippet from the component that displays details about an organization in the Polar Signals cloud product.

// /pages/organizations/[organizationId]/index.tsx
import { useState} from 'react';
import {Button, Input} from '@parca/components';
import {User} from '@polarsignals/client-grpc-web/polarsignals/user/user';
import type {NextPage} from 'next';
import {useRouter} from 'next/router';
import {useOrganizationUsers} from 'hooks/data/useOrganizations';

const Organization: NextPage = () => {
  const router = useRouter();
  const {organizationId} = router.query;
  const {data, mutations} = useOrganizationUsers(
    organizationId
  );

  const [newUserEmail, setNewUserEmail] = useState<string>(“”);

  const deleteUser = (user: User) => {
    mutations.deleteOrganizationUser(organizationId, user.id);
  };

  const addUser = (email: string) => {
    mutations.addOrganizationUser(organizationId, email);
  };

  return (
    // …
  );
};

export default Organization;

Using the action hook in a presentational component like the one above is straightforward as you can see. We simply import the `useOrganizationUsers` hook and use the destructing pattern to get the data and the mutations.

For the mutation functions, we simply wrapped them in local functions that will be triggered after a button click. Because of the `refetch` function in the mutation functions, the API knows to fetch the query again and re-render the UI if there’s a change in data.

The `deleteUser` function accepts a user property, which has a type that was imported from the client file at the top of the file. Because gRPC generates our client files in TypeScript, we can easily import types from the client and use that in the codebase as opposed to manually writing types for responses from the API for usage in the presentational components.

---

If you’ve made it this far, thanks for reading this article on how we use gRPC to build our frontend here at Polar Signals. So far this architecture has worked for us and helped to achieve seamless data fetching that automatically updates the UI with the latest data after a mutation.

If you have any questions or suggestions on how we use gRPC, and gRPC-web in our frontend, feel free to join our Discord community server. You can also star the Parca project on GitHub and contributions are welcome!