Endless Scrolling with AdapterViews and RecyclerView

Overview

A common application feature is to load automatically more items as the user scrolls through the items (aka infinite scroll). This is done by triggering a request for more data once the user crosses a threshold of remaining items before they’ve hit the end.

The approaches for ListView, GridView and RecyclerView (the successor to ListView) are documented here. Both are similar in code except that the LayoutManager in the RecyclerView needs to be passed in to provide the necessary information to implement infinite scrolling.

In both cases, the information needed to implement the scrolling include determining the last visible item within the list and some type of threshold value to start fetching more data before the last item has been reached. This data can be used to decide when to load more data from an external source:

To provide the appearance of endless scrolling, it’s important to fetch data before the user gets to the end of the list. Adding a threshold value therefore helps anticipate the need to append more data.

NOTE: This page is describing a straightforward yet a bit dated solution. For Android University courses it’s recommended to use this approach for simplicity. For more advanced cases, Google released its own Paging Library. You can see this guide for more information.

Implementing with RecyclerView

We can use a similar approach with the RecyclerView by defining an interface EndlessRecyclerViewScrollListener that requires an onLoadMore() method to be implemented. The LayoutManager, which is responsible in the RecyclerView for rendering where items should be positioned and manages scrolling, provides information about the current scroll position relative to the adapter. For this reason, we need to pass an instance of what LayoutManager is being used to collect the necessary information to ascertain when to load more data.

Implementing endless pagination for RecyclerView requires the following steps:

Copy over the EndlessRecyclerViewScrollListener.java into your application. It is a Java class, but it can be subclassed from Kotlin code as shown below thanks to Kotlin’s Java interop.
Call addOnScrollListener(...) on a RecyclerView to enable endless pagination. Pass in an instance of EndlessRecyclerViewScrollListener and implement the onLoadMore which fires whenever a new page needs to be loaded to fill up the list.
Inside the aforementioned onLoadMore method, load additional items into the adapter either by sending out a network request or by loading from another source.

To start handling the scroll events for steps 2 and 3, we need to use the addOnScrollListener() method in our Activity or Fragment and pass in the instance of the EndlessRecyclerViewScrollListener with the layout manager as shown below:

class MainActivity : AppCompatActivity() {
    // Store a member variable for the listener
    private lateinit var scrollListener: EndlessRecyclerViewScrollListener

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)
        // Configure the RecyclerView
        val rvItems = findViewById<RecyclerView>(R.id.rvContacts)
        val linearLayoutManager = LinearLayoutManager(this)
        rvItems.layoutManager = linearLayoutManager
        // Retain an instance so that you can call `resetState()` for fresh searches
        scrollListener = object : EndlessRecyclerViewScrollListener(linearLayoutManager) {
            override fun onLoadMore(page: Int, totalItemsCount: Int, view: RecyclerView) {
                // Triggered only when new data needs to be appended to the list
                // Add whatever code is needed to append new items to the bottom of the list
                loadNextDataFromApi(page)
            }
        }
        // Adds the scroll listener to RecyclerView
        rvItems.addOnScrollListener(scrollListener)
    }

    // Append the next page of data into the adapter
    // This method probably sends out a network request and appends new data items to your adapter.
    fun loadNextDataFromApi(offset: Int) {
        // Send an API request to retrieve appropriate paginated data 
        //  --> Send the request including an offset value (i.e `page`) as a query parameter.
        //  --> Deserialize and construct new model objects from the API response
        //  --> Append the new data objects to the existing set of items inside the array of items
        //  --> Notify the adapter of the new items made with `notifyItemRangeInserted()`
    }
}

Resetting the Endless Scroll State

When you intend to perform a new search, make sure to clear the existing contents from the list and notify the adapter the contents have changed as soon as possible. Make sure also to reset the state of the EndlessRecyclerViewScrollListener with the resetState method:

// 1. First, clear the array of data
listOfItems.clear()
// 2. Notify the adapter of the update
recyclerAdapterOfItems.notifyDataSetChanged() // or notifyItemRangeRemoved
// 3. Reset endless scroll listener when performing a new search
scrollListener.resetState()

You can refer to this code sample for usage and this code sample for the full endless scroll source code.

All of the code needed is already incorporated in the EndlessRecyclerViewScrollListener.java code snippet above. However, if you wish to understand how the endless scrolling is calculated, the detailed explanation is available here.

Troubleshooting

If you are running into problems, please carefully consider the following suggestions:

For the ListView, make sure to setup the setOnScrollListener listener in the onCreate method of the Activity or onCreateView in a Fragment and not much later otherwise you may encounter unexpected issues.
In order for the pagination system to continue working reliably, you should make sure to clear the adapter of items (or notify adapter after clearing the array) before appending new items to the list. For RecyclerView, it is highly recommended to make more granular updates when notifying the adapter. See this video talk for more context.
In order for this pagination system to trigger, keep in mind that as loadNextDataFromApi is called, new data needs to be appended to the existing data source. In other words, only clear items from the list when on the initial “page”. Subsequent “pages” of data should be appended to the existing data.
If you see Cannot call this method in a scroll callback. Scroll callbacks might be run during a measure & layout pass where you cannot change the RecyclerView data., you need to do the following inside your onLoadMore() method as outlined in this Stack Overflow article to delay the adapter update:
```
// Delay before notifying the adapter since the scroll listeners
// can be called while RecyclerView data cannot be changed.
view.post {
    // Notify adapter with appropriate notify methods
    adapter.notifyItemRangeInserted(curSize, allContacts.size - 1)
}
```

Displaying Progress with Custom Adapter

To display the last row as a ProgressBar indicating that the ListView is loading data, we do the trick in the Adapter. Having defined two types of views in getItemViewType(int position), we can display the last row differently from a normal data row. It can be a ProgressBar or some text to indicate that the ListView has reached the last row by comparing the size of data List to the number of items on the server side. See this gist for sample code.

Using Uncover library

Warning: the Uncover library is unmaintained — its last release on Maven Central was 2.0.1 in June 2017. For new projects, prefer the Jetpack Paging library covered in the Paging Library Guide (see also the NOTE at the top of this page). This section is kept for legacy reference only, and its snippets remain in Java since the library exposes a Java callback API.

If you use RecyclerView to implement ‘infinite scrolling’ over output of the web service, or some other resource that requires slow background calls with multiple items at once, it may be reasonable to try the Uncover library from Maven central:

dependencies {
    implementation ('io.github.andviane:uncover:2.0.1@aar')
}

This library requires you to implement the primary data fetcher, mediating between fast single item UI-thread calls on model and slow chunked calls on the background on your fetcher:

    final RecyclerView recyclerView = (RecyclerView) findViewById(R.id.recycler);
    final UncoveringDataModel<String> model = new UncoveringDataModel<>();

    model.setPrimaryDataProvider(new PrimaryDataProvider<String>() {

      @Override
      public PrimaryResponse fetch(PrimaryRequest primaryRequest) {
        Log.i("Fetch", "Service call to fetch items" + 
          primaryRequest.getFrom() + "- " + primaryRequest.getTo());
        ...
        ArrayList<String> data = new ArrayList<String>();
        ...
        return new PrimaryResponse<String>(data, Integer.MAX_VALUE);
      }
    });

    RecyclerView.Adapter adapter = new RecyclerView.Adapter() {
      public int getItemCount() {
        return model.size();
      ...
    };
    model.install(recyclerView, adapter);

The model.install glues model, view and adapter into working implementation. When the user swipes forward quickly, the library skips unneeded fetches of data between the new and old position, last requested data are fetched first, and any pending requests are dropped from the queue if they data are no longer visible.

Implementing with ListView or GridView (deprecated)

See Implementing with RecyclerView section for the most recent and relevant instructions. ListView is no longer used in modern Android applications.

Every AdapterView (such as ListView and GridView) has support for binding to the OnScrollListener events which are triggered whenever a user scrolls through the collection. Using this system, we can define a basic EndlessScrollListener which supports most use cases by creating our own class that implements OnScrollListener:

import android.widget.AbsListView

abstract class EndlessScrollListener(
    // The minimum number of items to have below your current scroll position
    // before loading more.
    private val visibleThreshold: Int = 5,
    // Sets the starting page index
    private val startingPageIndex: Int = 0
) : AbsListView.OnScrollListener {
    // The current offset index of data you have loaded
    private var currentPage = startingPageIndex
    // The total number of items in the dataset after the last load
    private var previousTotalItemCount = 0
    // True if we are still waiting for the last set of data to load.
    private var loading = true

    override fun onScroll(view: AbsListView, firstVisibleItem: Int, visibleItemCount: Int, totalItemCount: Int) {
        // If the total item count is zero and the previous isn't, assume the
        // list is invalidated and should be reset back to initial state
        if (totalItemCount < previousTotalItemCount) {
            currentPage = startingPageIndex
            previousTotalItemCount = totalItemCount
            if (totalItemCount == 0) {
                loading = true
            }
        }
        // If it's still loading, we check to see if the dataset count has
        // changed, if so we conclude it has finished loading and update the current page
        // number and total item count.
        if (loading && totalItemCount > previousTotalItemCount) {
            loading = false
            previousTotalItemCount = totalItemCount
            currentPage++
        }

        // If it isn't currently loading, we check to see if we have breached
        // the visibleThreshold and need to reload more data.
        // If we do need to reload some more data, we execute onLoadMore to fetch the data.
        if (!loading && firstVisibleItem + visibleItemCount + visibleThreshold >= totalItemCount) {
            loading = onLoadMore(currentPage + 1, totalItemCount)
        }
    }

    // Defines the process for actually loading more data based on page
    // Returns true if more data is being loaded; returns false if there is no more data to load.
    abstract fun onLoadMore(page: Int, totalItemsCount: Int): Boolean

    override fun onScrollStateChanged(view: AbsListView, scrollState: Int) {
        // Don't take any action on changed
    }
}

Notice that this is an abstract class, and that in order to use this, you must subclass this base class and define the onLoadMore method to actually retrieve the new data. The three overloaded Java constructors from the original version are expressed here as constructor parameters with default values. We can define an anonymous object that extends EndlessScrollListener within any activity and bind that to the AdapterView. For example:

class MainActivity : AppCompatActivity() {
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        // ... the usual
        val lvItems = findViewById<ListView>(R.id.lvItems)
        // Attach the listener to the AdapterView onCreate
        lvItems.setOnScrollListener(object : EndlessScrollListener() {
            override fun onLoadMore(page: Int, totalItemsCount: Int): Boolean {
                // Triggered only when new data needs to be appended to the list
                // Add whatever code is needed to append new items to your AdapterView
                loadNextDataFromApi(page)
                // or loadNextDataFromApi(totalItemsCount)
                return true // ONLY if more data is actually being loaded; false otherwise.
            }
        })
    }

    // Append the next page of data into the adapter
    // This method probably sends out a network request and appends new data items to your adapter.
    fun loadNextDataFromApi(offset: Int) {
        // Send an API request to retrieve appropriate paginated data 
        //  --> Send the request including an offset value (i.e `page`) as a query parameter.
        //  --> Deserialize and construct new model objects from the API response
        //  --> Append the new data objects to the existing set of items inside the array of items
        //  --> Notify the adapter of the new items made with `notifyDataSetChanged()`
    }
}

Now as you scroll, items will be automatically filling in because the onLoadMore method will be triggered once the user crosses the visibleThreshold. This approach works equally well for a GridView and the listener gives access to both the page as well as the totalItemsCount to support both pagination and offset based fetching.