I totally get this frustration—balancing plot marker size (for readability with 5000 points) and legend marker size (so users can actually see them) is tricky, especially when plotlyProxy messes up your custom setup. Let's break down two solutions that fix this issue:

Solution A: Target Only the Color Property in plotlyProxy #

The root problem with your current code is that when you use list(marker = list(color = input$markercolor)) in plotlyProxyInvoke, you're replacing the entire marker object for the trace—including the custom size array you built to trick the legend into being larger. Instead, you can update just the color property of the existing marker object, leaving the size array intact.

Modified Server Code for Solution A #

server <- function(input, output, session) {
  observeEvent(input$Switch, {
    plotlyProxy("plot1", session) %>%
      # Update ONLY the marker.color, not the entire marker object
      plotlyProxyInvoke("restyle", list('marker.color' = input$markercolor), list(as.numeric(input$traceNo)-1))
  })

  output$plot1 <- renderPlotly({
    markersize <- 4
    markerlegendsize <- 20
    colors <- c('red', 'blue', 'black')
    p1 <- plot_ly()
    p1 <- add_trace(p1, data = mtcars, x = ~disp, y = ~mpg, type = 'scatter', mode = 'markers', color = ~as.factor(cyl), colors = colors)
    p1 <- layout(p1, title = 'mtcars group by cyl with switching colors')
    p1 <- plotly_build(p1)
    # Keep your existing legend size hack
    for(i in seq(1, length(sort(unique(mtcars$cyl) )))) {
      length.group <- nrow(mtcars[which(mtcars$cyl == sort(unique(mtcars$cyl))[i]), ])
      p1$x$data[[i]]$marker$size <- c(rep(markersize,length.group), rep(c(-markersize+2*markerlegendsize), length.group))
    }
    p1
  })
}

Why This Works #

By using the string syntax 'marker.color', you're telling Plotly to update only that specific property, not overwrite the entire marker structure. Your custom size array stays in place, so the legend keeps its larger marker size while the plot markers remain small, even after triggering plotlyProxy.

Solution B: Use JavaScript to Force Legend Marker Size (Independent of Plot Markers) #

If you want a more robust solution that completely decouples legend marker size from plot markers (no more hacks with size arrays), you can use JavaScript to directly modify the legend SVG elements. This way, no matter what changes you make via plotlyProxy, the legend markers stay the size you want.

Modified Code for Solution B #

First, add a JavaScript snippet to your UI that runs after the plot renders:

ui <- fluidPage(
  tags$head(
    tags$script(HTML("
      // Wait for the plot to render, then adjust legend markers
      $(document).on('shiny:value', function(event) {
        if (event.target.id === 'plot1') {
          setTimeout(function() {
            // Set your desired legend marker size here (in pixels)
            const legendMarkerSize = 20;
            // Target all legend marker circles
            $('#plot1 .scatterlayer .trace .legendpoints circle').each(function() {
              $(this).attr('r', legendMarkerSize);
            });
          }, 100);
        }
      });
      
      // Also re-adjust after plotlyProxy updates
      Shiny.addCustomMessageHandler('updateLegendSize', function(message) {
        const legendMarkerSize = 20;
        $('#plot1 .scatterlayer .trace .legendpoints circle').each(function() {
          $(this).attr('r', legendMarkerSize);
        });
      });
    "))
  ),
  fluidRow(
    column(8, plotlyOutput("plot1")),
    column(2,
           colourpicker::colourInput(inputId = 'markercolor', label = 'X', palette = "limited", showColour = "background", returnName = TRUE),
           selectInput(inputId = 'traceNo', label = 'Trace', choices = c(1:3), selected = 1),
           br(),
           h5('Switch'),
           actionButton(inputId = 'Switch', label = icon('refresh'), style = "color: #f7ad6e; background-color: white; border-color: #f7ad6e; height: 40px; width: 40px; border-radius: 6px; border-width: 2px; text-align: center; line-height: 50%; padding: 0px; display:block; margin: 2px")
    )
  )
)

Then update the server to trigger the JavaScript after plotlyProxy changes:

server <- function(input, output, session) {
  observeEvent(input$Switch, {
    plotlyProxy("plot1", session) %>%
      plotlyProxyInvoke("restyle", list(marker = list(color = input$markercolor)), list(as.numeric(input$traceNo)-1))
    
    # Tell JavaScript to re-adjust legend markers after the restyle
    session$sendCustomMessage("updateLegendSize", list())
  })

  output$plot1 <- renderPlotly({
    markersize <- 4
    colors <- c('red', 'blue', 'black')
    p1 <- plot_ly()
    p1 <- add_trace(p1, data = mtcars, x = ~disp, y = ~mpg, type = 'scatter', mode = 'markers', color = ~as.factor(cyl), colors = colors, marker = list(size = markersize))
    p1 <- layout(p1, title = 'mtcars group by cyl with switching colors')
    p1
  })
}

Why This Works #

• The first JavaScript function runs when the plot initially renders, setting all legend marker circles to your desired size.
• The custom message handler triggers after each plotlyProxy restyle, ensuring the legend markers stay the right size even after color changes.
• You no longer need the size array hack—plot markers are just set to your small size, and legend markers are controlled directly via SVG attributes.

Which Solution Should You Choose? #

• Solution A is quick and dirty, perfect if you want to keep your existing code structure with minimal changes. It relies on Plotly's property-targeting syntax in restyle.
• Solution B is more maintainable and robust, especially for large datasets or complex apps. It completely separates plot and legend marker sizes, no hacks required.

内容的提问来源于stack exchange，提问作者Mark